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I. INTRODUCTION 
Harotp A. EDGERTON AND WILLIAM J. JONES 


Learning by doing is a frequently indorsed 
but seldom practiced principle of teaching at 
the college level. While it is not uncommon 
for teachers to lecture for hours on the value 
of direct experience, few actually carry on in- 
struction in their classes in terms of real 
problems. The values which accrue from such 
direct experience cannot be evaluated solely 
in terms of the products of such experiences 
for the very act of participating in the 
Processes involved in direct experience has 


unique values which need to be considered in 
judging the worthwhileness of learning by 
doing. 

The series of articles which follow are the 
products of a class in statistics at Ohio State 
University which tackled real problems in 
terms of actual data and arrived at conclu- 
sions which were of importance to college 
officials. These articles do not constitute 
even the principal source of data for evaluat-. 
ing the instructor’s hypothesis that students 
are likely to become better statisticians if 
they work on practical problems, but they 
are offered as partial evidence of the worth 
of such an approach to the teaching of 
statistics. Other factors which need to be 
considered in appraising the value of this 
“problem method” of teaching statistics are 
the manipulative skills which were developed 
with respect to the use of the Hollerith ma- 
chine, the techniques of setting up procedures 
for collecting and organizing raw data, the 
administrative techniques of actually organ- 
izing and carrying through a study on sched- 
ule, and the ability to write an account of 
the study. 


The following articles should be of inter- 
est not only because they are the product of 
a class, but because they constitute a rather 
comprehensive study of a series of five evalu- 
ation devices which were used in conjunction 
with an orientation course for college 
freshmen. 

In 1937, the College of Education at the 
Ohio State University inaugurated an experi- 
mental freshman orientation course called 
Survey of Education 407. One of the prin- 
cipal assumptions which operated to direct 
the organization of the course was that the 
personnel work and the instructional activity 
of the College needed to be more closely in- 
tegrated. Accordingly, all of the students 
taking the course were assigned on the basis 
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of their proposed teaching field interests (if 
any) to advisory groups of about 15 to 20. 
The students met twice a week in their small 
advisory sections and twice a week in much 
larger lecture sections. 

In order to facilitate the guidance work 
which was carried on and in order to provide 
the students with more information about 
themselves with respect to a wide variety of 
aspects of personality, several testing devices 
were given to the students during the course. 
Since the objectives of the college program 
include a concern for interests, attitudes, 
habits of thinking, and several other aspects 
of personality, in addition to subject matter 
accomplishment, many evaluation instruments 
designed to measure such objectives were 
made features of the testing program of the 
course. 

A veritable wealth of information was thus 
available, and constituted the raw data for 
the studies which follow. The instructor and 
members of the class acknowledge the kind- 
ness of the office of the Junior Dean of the 
College of Education and the Evaluation Di- 
vision of the Bureau of Educational Research 
for their cooperation in making the data for 
these studies available. 

Five evaluation instruments were singled 
out for study by the members of the class 
as follows: 


. Contemporary Affairs Test (1938 form) 
. Interpretation of Data Test 

. Interest Questionnaire 

. Social Problems Test 

Social Attitudes Scale 


Mmhwn eae 


All of these testing devices, with the ex- 
ception of the Contemporary Affairs test, 
were developed by the Evaluation Staff of 
the Eight Year Study of the Progressive Edu- 
cation Association. The Contemporary Af- 
fairs test was developed by the Cooperative 
Test Service. Each of these tests is described 
in the studies which follow. 

A unique feature of the following articles 
is that they all have exactly the same in- 
dividuals included in the statistical samples 
used. The 462 cases which constitute the 
sample for one study make up the sample 
for every other study in the series. In fact, 
the 462 cases for these studies were selected 
on just such a basis of communality. In ad- 
dition to the test data mentioned above, as 
much of the following information was col- 
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lected about each of the 524 autumn quarter 
1938 freshmen as was possible by consulting 
available university records: 


. Age at time of entrance to college 

. Size of high school graduating class 

. Rank in high school graduating class 

. Father’s education 

. Mother’s education 

Father’s occupation 

. Best liked high school subject 

. Least liked high school subject 

. Ohio State University Psychological 
Examination. 

. Reading test score on Ohio State Up). 
versity Psychological Examination 

11. English placement test score 

12. Troyer Index* 

13. Autumn quarter point-hour ratio 

14. Winter quarter point-hour ratio 

15. Cumulative point-hour ratio 

16. Grades in various courses 


“An average of the student’s reading percentil 
gence test percentile, and an application blank ; 
based on 19 factors. 


OPAMP W 


° 


When all of the above data for the entire 
group of 1938 autumn quarter freshmen were 
collected, they were examined for complete- 
ness. With respect to items 14 and 15 above 
the data were found to be more incomplete 
than they were with respect to the other factors 
listed. Although from the outset it seemed 
desirable to the class to work toward the end 
of having an interlocking set of studies based 
on identical cases, and although it would 
have been desirable to have more common 
data such as those listed above to relate to 
the test variables, yet it seemed that 2 
definite bias would have been introduced into 
the sample if all those cases were rejected 
from inclusion in the sample for which there 
were no winter quarter or cumulative poin! 
hour ratios available. A definitely scholasti- 
cally inferior group of students would have 
been thus rejected, and the remaining sample 
would not have been as typical of entering 
autumn quarter freshmen. Thus, while the 
criteria for inclusion in the sample which was 
finally selected could have been chosen 
terms of completeness of the data with 
respect to most if not all of the sixteen {ac 
tors jisted above, (and it would have been 
desirable to have had more such common 10 
formation to correlate with the test var 
ables), yet in the interests of selecting 4 
typical sample, fewer criteria were applied 
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Only those cases have been rejected from the 
© sample for which intelligence test data, 
autumn quarter point hour ratios, and scores 

» all five of the tests which were selected 
were incomplete. Four hundred sixty-two of 
the cases had complete data for these seven 
‘actors and thus constitute the common 
sample for all of the studies which follow. 

In order to characterize better the sample 
ysed in these studies, it may be noted that 
‘the 462 students, 305 were girls and 157 
were boys. This is a ratio of girls to boys 
‘ almost exactly two to one. A study of the 
racial composition of the group indicated 
that the sample contained 437 white cases 
and 25 colored cases, or 94% white and 
6% colored. 

Table I shows the distribution of ages, 
measured from the last birthday, at the time 
if entrance to college of the students com- 
posing the samples. It should be noted that 
over half of the group was 18 years old at 
the time of entrance to college. Further evi- 
dence of the age homogeneity of the group is 
the fact that 85% of the group was between 
_ 17 and rg years of age at the time of entrance 
to college. 

One way to characterize the size of the 
communities from which students come is to 
use the size of the high school graduating 
lass as a rough index, for usually the larger 
graduating classes are found in the larger 
communities. Mid-year graduating classes 
and consolidated schools’ classes, however, 
are exceptions to the generalization. Table IT 
indicates the size of the graduating class of 
the students composing the sample. It will be 
noted that the group is not homogeneous in 


TABLE I 


AGE _ DISTRIBUTION OF 462 OHIO STATE 
UNIVERSITY EDUCATION FRESHMEN 
AT TIME OF ENTRANCE 


Autumn, 1938 


Year of Birth Number Per Cent 
ff ae 9 2 
93 eiaietasaiiies tn 5 1 
_ Pe 10 2 
21 <b acaxe eee 18 4 
_—_ OE NT 25 5 
fa SCE 71 15 
_ RARE 252 55 
_ EIS 8.8 67 15 
19 OF younger _...._______ 5 1 
SN ee ee 462 100 
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TABLE II 


Size OF HIGH SCHOOL GRADUATING CLASS OF 
AUTUMN QUARTER EDUCATION FRESHMEN 
1938 


Size of Class Number Per Cent 


Gai of more ..._.......... 3 1 
I 73 16 
ID eiscsestcecrasiscsiemusavenniencsctsean aE 13 
159-251 __ ee ein 10 
100-158 __ eR es 10 
64-— 99 __ pene iP = ae 8 
_{). eS _. 45 10 
26- 39 _ SED Spe 34 s 
16— 25 _ dae OR 38 8 
cS) ae 23 5 
No data _ 52 11 

Totals _ 462 100 


this respect, for the distribution has high 
variability. The median size of high school 
graduating class was 100, while the step in- 
terval having the largest frequency is 399—- 
630. These data seem to point to the fact. 
that the students come from schools with 
large graduating classes, and probably from 
large communities more often than from 
small or rural communities. 

Over one-half of the freshmen studied 
ranked in the upper third of their high school 
graduating classes, while 33% ranked in the 
middle third, and 12% ranked in the lower 
third. 

Table III presents the extent of the fathers’ 
and mothers’ education of the students who 
were studied. Slightly more than one-half of 
both the fathers and mothers have an edu- 
cation equivalent to high school education 
or better although about three-fourths of the 
parents have not attended college. More 
fathers than mothers have been graduated 
from college, the figures being 16% and 5% 
respectively. Approximately one-fourth of the 
students’ fathers and mothers have had only 
eighth grade education or less. The two-way 
table likewise reveals that nearly half the 
fathers and mothers have had the same 
amount of education and that in cases where 
the parents have had unequal amounts of 
education, the father has had more education 
than the mother two times out of three. 
Column 1 in ‘\'able IIT reveals the interesting 
marital combination of a woman college 
graduate with a man who had no more than 
eighth grade education. 

The distribution of fathers’ occupations ac- 
cording to the United States Employment 
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TABLE III 


EXTENT OF FATHERS’ AND MOTHERS’ EDUCATION OF 462 OHIO STATE UNIveERsity 
COLLEGE OF EDUCATION FRESHMEN 


Autumn, 1938 


Fathers’ Education 


(1) College Graduate ee a ee 
(2) College, but not Graduate ~~ ateeilanen 
(3) High School Graduate : 

(4) High School, but not Graduate --.----- 
(5) Eighth Grade or Less ~------~-~- 


(6) No data ..-......--.------ 


Totals 
Per Cent a 


Service’s classification is shown in Table IV. 
The data indicate that the students come 
largely from the professional, sales, and 
skilled craftsmen classes of workers, nearly 
two-thirds of the fathers being classified in 
these groups. 

The distribution of the intelligence test 
scores of the education students who were 
made the objects of the studies which follow 
is presented in Table V. The skewness of the 
distribution toward the upper end may be 
attributable to the fact that until recent 
years a selective admission policy was opera- 
tive in the College of Education with an in- 
telligence percentile rank of 30 being regarded 
as a critical point. While this selective ad- 
mission on a basis of intelligence is no longer 
in force, secondary school officials may yet 
be making it one of the bases for recom- 


TABLE IV 


FATHERS’ OCCUPATIONS OF 462 OHIO STATE 
UNIVERSITY COLLEGE OF EDUCATION 
FRESHMEN 


Autumn, 1938 


(According to the United States Employment 
Service Classification) 


Occupation Number Per Cent 
Professional and kindred __ 108 23 
Sales persons ~....___-_-- 93 20 
Clerical workers _...__._... 35 8 
Service workers _._______- 26 6 
Skilled craftsmen’ ________ 113 24 
Production workers —__-__— 27 6 
Physical laborers _________ 25 5 
Degen .............. 9 2 
i ee See 26 6 

re 462 100 


*Includes farmers. 





Mothers’ Education Totals Per Cep: 

(1) (2) (8) (4) 6 
15 26 19 5 3 5 7 16 

2 23 15 5 7 0 52 11 

2 16 7 1318 3 121 26 

2 8 10 30 19 4 73 16 

1 7 2 19 64 3 116 25 

3 2 ¢ 2 3 #14 27 8 
25 82 143 74 109 29 462 

5 18 31 16 24 6 100 


TABLE V 


DISTRIBUTION OF INTELLIGENCE Test Percey- 


TILES FOR 462 On1o STATE UNIVERsITY 
COLLEGE OF EDUCATION FRESHMEN 


Autumn, 1938 


Percentiles Number Per Cen: 
ee eae 75 16 
2 ae eee 62 13 
i ee See: 65 14 
_, 2 eae ee oe 64 $ 
a 45 10 
7 Rn ee 46 10 
a ee eee 41 ’ 
= See eae 21 ; 
TS eee ee 25 5 

Oa 18 4 
IE hak: Sets a 462 100 


mendation to college, and thus a once- 
removed selective admission policy might be 


in effect. It should be pointed out, too, that 
the only cases which were included in th 
sample were those for which autumn quarter 
point hour ratios were available, and that the 
application of this criterion served to reject 
from consideration many students wh 
dropped out of school before the end of the 
first quarter. Probably more students wit) 
low intelligence test scores than students wit) 
high intelligence test scores dropped out be- 
cause they were unable to meet scholastic 
requirements during the first quarter. This 
may be another possible explanation of the 
skewness. A third possibility is that the 
changing character of entering freshmen from 
year to year has rendered the norms less 
stable and less comparable. It is probably 
true that the sample selected for study was 
not different from the entire autumn quarter 
education freshman group of 1938. 





VWarck, 1940] 


11 A STUDY OF A TEST FOR 
MEASURING SKILL IN THE 
INTERPRETATION OF DATA 


Wape S. AMSTUTZ 
Rupert C. KOENINGER 


The Interpretation of Data test used in the 
freshman testing program was designed to 
test the students’ ability to analyze data and 
vet meaning from them. The test consists of 
ten sets of problem data about which the stu- 
dent is asked to make interpretations. Each 
set of data has a variety of statements which 
the students are asked to interpret as true, 
probably true, insufficient data, probably 
false. and false. To give a correct answer to 
all these involves skill in recognizing state- 
ments, summarizing the data, contrasting 
portions of the data either by regrouping the 
data or by translating the relationships into 
easier terms, and recognizing high and low 
points as well as points of change. To do 
this involves an ability to recognize the 
tacts. 

The test is also designed to measure ability 
in drawing interpretations and _ projecting 
meanings. What meanings have the facts 
presented in the problems? One set of con- 
clusions are somewhat beyond the facts but 
in general are in agreement with a definitely 
established trend. Another set of inferences 
concern points not actually presented in the 
data and, at the same time, are consistent 
with an established trend. In the former we 
have “prudent extrapolation” while the latter 
involves “prudent interpolation”. 

A third characteristic is concerned with 
the ability to recognize the limitations of the 
data under consideration. There are instances 
in which there is insufficient evidence to war- 
rant a given statement. If a statement is 
made and not supported by the evidence, the 
student is going beyond the available data. 

A more detailed analysis shows five kinds 
ol statements in the test. The first type are 
those which are rather fully supported by the 
data. They may be statements which sum- 
marize the data, statements which indicate 
trends, statements which make new groupings 
of data, or translate relationships into easier 
terms or ratios, statements which point out 
significant items, or statements which indicate 
the range of the data. All of the above types 
of statements should be marked “true” by 
the students. 
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The second type of statements are those 
which may be thought of as only “probably 
true” and which are partially supported by 
the evidence. Included in these are state- 
ments which are slightly beyond the facts but 
are consistent with all that is known about 
the data, statements which involve points 
within a trend but are not established by the 
data, and those which draw conclusions about 
larger groups on the basis of adequate 
sampling. 

The next type are statements which involve 
interpretations which are considered as “un- 
certain” because of the insufficient evidence. 
Included are statements which assign cause, 
predict effects or assign implication to the 
data, attribute purpose, extrapolate the 
trends too far, attribute values to outcomes, 
contain false analogies, make unjustified gen- 
eralizations, or make use of a “per cent- 
equals-number”’ fallacy. 


In the fourth category are those statements 
which are considered “probably false”. In 
this group are found statements which go 
slightly beyond the facts and are inconsistent 
with the data, those which involve points not 
established by the data, which are incon- 
sistent with a trend, and those statements 
which draw conclusions about larger groups 
which are contradicted by a sample. 


Finally, there are those statements about 
the data which are “false”. Included in these 
statements are those which are inaccurate 
summarizations of the data, contradict a given 
trend, make false contrasts and erroneous 
computations, present inaccurate ranges and 
are contrary to the evidence, contrast one 
trend with another inaccurately, and those 
which group the data erroneously. 

Faced with statements of the kinds indi- 
cated, it was the student’s task to mark each 
of them as either true, probably true, in- 
sufficient data, probably false, or false. With 
such a set of student responses, eight differ- 
ent kinds of scores were obtained on the test. 
Hereafter they will be referred to as variables 
1 to 8. 


(1) General accuracy 

(2) Accuracy with probably true and 
probably false statements 

(3) Accuracy in recognizing statements 
— have insufficient data to support 
them 





* 
- 


th 
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(4) Accuracy with true and false state- 
ments 

(5) Score indicating omissions 

(6) Caution 

(7) Beyond the facts 

(8) Crude errors 


The general accuracy score (variable 1) 
indicates the students’ ability to mark all 
five types of statements correctly. This score 
is obtained by totaling the number of insuffi- 
cient evidence items marked uncertain, the 
true items marked true, the probably true 
items marked probably true, the probably 
false items marked probably false, and the 
false items marked false. (In this study all 
scores are given in per cent of possible score 
in that classification.) 

The second score indicates the general ac- 
curacy with which the student judged only 
those statements which are probably true or 
probably false. The third indicates the stu- 
dents’ ability in correctly recognizing state- 
ments which are not supported by the evi- 
dence available. The fourth is a score of the 
per cent of true and false statements which 
a student marks correctly. Provision is made 
for a fifth score indicating the number of 
omissions students made. 

The sixth is a caution score indicating to 
what extent the student had a tendency to 
judge statements with less certainty than the 
data warrant. If a student marked a true 
statement as probably true, he is scored 
“cautious”. If he marked a probably true or 
probably false statement as insufficient data, 
he was likewise scored “cautious”’. 

A common type of student error is that of 
going beyond the facts as presented in the 
data. The seventh score indicates the extent 
of this kind of error. If a student marked a 
probably true or probably false item as true 
or false, he was going beyond the data. The 
same is true if he marked any of the insuffi- 
cient data statements as true, probably true, 
false, or probably false. 

The eighth score is called crude errors and 
gives an indication of the students’ tendency 
to make wide errors in judgment. For 
example, if a statement is marked false when 
the information would support it as true, or 
true when the information would indicate it 
as false, he is making a crude error. 

Table I presents a summary of the vari- 
ables in terms of their sources. For example, 
Table I shows that if a true statement were 
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TABLE I 
A SUMMARY OF THE SCORING Scueny 


Statements which 


should have been If marked by this 


marked student 
T PT ID PF 
SS a | 6 x 
PT -— — 7 (2) 6 x 
ID ae. t «6 (3) — 7 
PF ener esrerenes S 8 6 (2) 
F eee s 8 a t 4 


* Those variables in parenthesis contriby: 
to the general accuracy score as 
those indicated by the number. 


marked true, it would contribute to the : 
on variable 4, a correct response. If a try 
statement were marked probably true or ip. 
sufficient data, it would contribute to th 
score on variable 6, a caution error, and 
it were marked probably false or false, ; 
would contribute to the score on variable ‘ 
a crude error. 

It is believed that true and false statements 
are primarily ones that require only good 
reading habits. One can look at the data and 
see whether or not the statement is true or 
false. The probably true and probably fale 
statements present a slightly more difficult 
situation. There are two main usages of the 
“probably true” statement. One of these is 
the projection of a trend, either forward or 
backward, and the other is a statement o! 
sampling. If it is true of this particular grou 
in question, then it is probably true of an- 
other or a larger group. In short, the range 
of statements that are possible is limited in 
the area of probably true and probably ‘ake 
statements. With the insufficient data state- 
ments the situation is different. There ae 
many ways of going beyond the data. for 
example, one may attribute cause, propo 
value judgments, or assign implications 0 
cosmic purposes. In the trials with the test ! 
was found that there were more of this ‘2t- 
ter type of statement needed in the test 
order to make the test more reliable. 

The reliability of the test as determined 
giving it to 110 tenth grade students in three 
different schools is shown in Table Il 
The estimated reliability for this particu’ 
sample and the number of possible items 
each variable of the test are also included 
Table II. It is to be noted that the fifth var 


* Data furnished by the Evaluation Staff if the Eight 
Year Study, Progressive Education Association, ‘ae 
Chicago, Chicago, Illinois. 








0) 
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TABLE II 


repiMATED RELIABILITY COEFFICIENTS OF THE 
~""SevERAL SCORES ON THE INTERPRE- 
TATION OF DATA 


Number 10th 
of Grade Estimated 
Possible Coeffi- Coeffi- 


Va! il it 
Items cient cient* 
+ General accuracy 119 91 81 
Probably true and 
probably false. 29 51 .67 
Insufficient data. 57 91 .86 
4 True—False _--. 33 .85 65 
Omissions ----. 119 ao 79 
Caution _------. 62 72 .68 
Beyond facts _-__ 86 88 85 
» Crude errors — 62 87 45 


Fstimated by means of formula of Case IV 
developed in the article, Kuder, G. F., and 
Richardson, M. W., “The Theory of the Esti- 
mation of Test Reliability”, Psychometrika, 
Vol. 2, Sept. 1937, pp. 151-160, Case IV. 


ible is omitted from the tenth grade study. 
Frequently this variable is no longer used, as 
the general opinion seems to prevail that 
little if any pertinent information about the 
student is reflected by the number of his 
missions. 

The purpose of this inquiry is to study the 
Interpretation of Data test in terms of the 
intercorrelations of the eight scores made by 
the 462 College of Education freshmen. Is 
there a high or low correlation between this 
test and the scores made on the intelligence 
test? Does a significant correlation exist be- 
tween the scores made on the Interpretation 
of Data test and the grades the students made 
for the fall quarter? Is it possible that two 
parts of the test are measuring the same 
thing? Would the same purpose be served by 
an elimination of some parts of the test? In 
view of the increased time required for ad- 
ministration and scoring of this test, it would 
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be an economy of time and labor if the same 
information could be given by three or four 
variables instead of seven or eight. 

Certain statistical characteristics of each 
of the variables are shown in Table III. A 
casual examination of the scores on the parts 
of the test as shown in Table III indicates 
that some of the distributions are noticeably 
skewed. Coefficients of skewness were com- 
puted for all the test variables, using the 
3(M tS ) All of the test 
variables showed positively skewed distribu- 
tions except variables 1, 3, and 4. None of 
them, however, were what one would call 
“much skewed”’. 

The intercorrelations of the test variables 
and their correlation with four of the “com- 
mon data” variables listed in the introduction 
to this series of articles are shown in 
Table IV. 

In the correlations shown in Table IV, 
three stand much higher than the others. 
Variables 1 and 3 correlate 0.78; variables 1 
and 7 correlate — 0.77; and variables 3 and 
7 correlate — 0.88. Since variable 1 is the 
sum of variables 2, 3, and 4, a high positive 
relation is expected of that variable with its 
component parts and a negative correlation 
of perhaps some lesser magnitude with the 
other variables. This tendency should be 
present since it is desirable to have a high 
score on variables 1, 2, 3, and 4 and to have 
a low score on variables 5, 6, 7, and 8. 

It is interesting to note in Table IV that 
the correlation of variables 10, 11, and 12 
(intelligence test score, point hour ratio, and 
grade in Survey of Education) show almost 
the same correlation with each of the test 
variables. The greatest difference between 
any two r’s in a given column is 0.06. This 


formula sk = 


TABLE III 
AVERAGES, DISPERSION, AND SKEWNESS OF VARIABLES 


N = 462 Education Freshman Students 


Test Variable 
General accuracy 


. Insufficient Data 

. True—False 

. Omissions 

6. Caution _.... 

- Beyond facts _..-.._______ 
Crude errors 


tm © 


‘ 


- Probably true and probably false ___________ 


Standard 
Mean Median Deviation Skewness 
Ma 47.6 47.8 10.2 —.06 
iasuiina 25.8 25.3 13.7 11 
oe Ne 48.6 49.3 16.8 —.13 
seeihertinceak 59.9 61.4 14.1 —.32 
ai iscan asia 2.6 2.1 3.1 48 
pon deen 28.0 27.9 10.0 .03 
migainnises 44.9 44.4 13.4 ll 
LO 14.6 13.8 6.0 40 
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TABLE IV 


TABLE OF INTERCORRELATIONS OF THE TEST VARIABLES AND CORRELATIONS OF TEST 
VARIABLES AND CERTAIN REFERENCE VARIABLES 


254 
Variables 1 2 
1. General Accuracy ol 
2. Probably True and 
Probably False 31 
3. Insufficient Data_ _. .78 03 
4. True—False _ - ‘ . 51 «12 
5. Omissions ; =a .10 
6. Caution_______- 07 . 28 
7. Beyond Facts 77 08 
ft: aa . 56 .15 
9. Rank in High School 
Graduating Class 31 09 
10. Psychological Test .48 . 26 
11. Point Hour Ratio 
Autumn Quarter 49 25 
12. Survey of Education 
Required course . 46 25 


raises the question whether or not these three 
variables are measuring the same thing. Re- 
moving intelligence (variable 10) from the 
point hour ratio (variable 11) the residual 
was correlated with each of the test variables. 
These part correlations are shown in Table V. 
For the purpose of comparison the total cor- 
relations with point hour ratio are given. The 
data contained in Table V indicate that in- 
telligence and point hour ratio have a very 
great similarity, but the variables are not to 
be considered statistically equivalent. 
Another great similarity in coefficients oc- 
curs with respect to variables 3 and 7. By 
further examination of the test it is found 
that the student has only two possible an- 
swers for all “insufficient data’? statements. 
By definition he will either answer it correctly, 
or incorrectly as “beyond the facts’. Should 
one decide to remove one of these variables 
on the assumption that they measure the 
same thing, g contradiction may exist. For 


TABLE V 


CORRELATION OF THE TEST VARIABLES WITH 
Point Hour Ratio (INTELLIGENCE TEST 
Score REMOVED FrRoM Point Hour Ratio) 


Part Total 
Variable Correlation r 

1. General Accuracy ______ .255 494 
2. Probably True and Prob- 

aney Pele .......... .124 .258 

3. Insufficient Data _______ .145 .318 

4. True—False ______ ; .239 443 

SS ae —.042 —.055 

6. Caution —____- : —.127 —.199 

7. Beyond Facts __________ —.141 —.305 

8. Crude Errors _......... —.195 —.391 


3 4 5 6 7 
.78 . 51 —.17 07 By i 
-.03 2 ~.10 —. 28 O8 
.15 .14 45 88 
15 .15 . 65 07 
—.14 15 aa 04 07 
45 . 55 EI 49 ay 
- 88 -.07 —.04 . 49 P 
.39 . $1 .07 —.04 37 
26 31 -.09 —.08 22 
33 .41 .038 —.16 32 
.32 . 44 05 .20 31 19 
. 30 .38 07 10 30 


example, if variable 7 (beyond the facts 
were removed it would be impossible to score 
variable 3 “insufficient data’ when the stu- 
dent answers it incorrectly. Furthermore, 
since there are only two possibilities with this 
type of test item, i. e., correct or beyond the 
facts, one can represent this relation by « 
linear function and predict exactly one of the 
scores when the other is given. The reason 
the correlation between variables 3 and 7 is 
not perfect is due to the fact that there is a 
small probability for an answer beyond the 
facts in the probably true and probably fale 
categories. If the probably true and probab) 
false statements were removed from the tes 
the result would be a perfect correlation be- 
tween variables 3 and 7. 

Is it necessary to use all eight scores to 
have the essential information contained in 
the test as now scored? Considering the fact 
that there are several high correlations, 't 
would seem that variables 1, 3, and 7 reflect 
essentially the same information. These three 
variables also have somewhat similar correls- 
tions with other variables. 

Since no other such combinations were 4p- 
parent from casual inspection, factorial ana'y- 
sis was applied. Using Thurstone’s centroi¢ 
method, the loadings for four factors wet 
obtained. These are shown in Table VI. 

The three variables 1, 3, and 7 stand oul 
in the first factor with loadings of 0.945, 
0.869, and — 0.837 respectively. To thee 
might be added variable 8 (crude errors) 
with a loading of —o0.603. Thus the firs 
centroid factor is made up largely of variable 
1, general accuracy, and 3, recognition © 


o0o08@8 °° 8 = « 
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TABLE VI 
FACTOR LOADINGS 


Variables 


I 

1. General Accuracy -~--~---~--~------- 945 
» Probably True and Probably False -- _.178 

Insufficient TIE icine cesnciinitnecmntsitcicnte .869 
{, True—False ~---------------------- 362 
5. Omissions ~------------------------ —.173 
s. Caution _----------~- --------------- -210 
y Bevond a ee —.837 
ne —.603 


statements not supported by the available 
evidence. Large negative loadings in the first 
‘actor are found for variable 7, going beyond 
the facts, and for 8, crude errors. 


The second centroid factor has a 0.707 load- 
ing for variable 4, accuracy with true and 
false statements, and a —o0.716 for 6, over- 
cautiousness. These two are logically oppo- 
sites since 4 is the proportion of true and 
false statements correctly marked, while 6 is 
a measure of the tendency to judge state- 
ments with less certainty than the data 
warrant. 


It may be noted that variables 1, 3, 7, and 
3, which have high loadings for the first fac- 
tor, have rather inconsequential loadings for 
the second. The two variables with high load- 
ings in the second factor have low loadings on 
the first factor. None of the third and fourth 
factor loadings appear to be of great impor- 
tance. The dropping of the third and fourth 
factors does not leave out much common in- 
iormation, as evidenced by the relative mag- 
nitudes of the estimated communalities which 
indicate the amount of variance taken into 
account for two and for four factors. For 
the purposes of this study rotation of the 
axes seems unnecessary. 


Omitting from the discussion variable 5, 
missions, only variable 2, accuracy with 
probably true and probably false statements, 
appears to have much information not in- 
cluded in either of the first two factors. The 
estimates of reliability of this variable are 
°.§1 and 0.67, while the communality is only 
0.172, 

It might be feasible then for some purposes 
‘o obtain as a minimum only three or four 
scores for this test, rather than eight, with- 
out losing much of the information of the 





Estimate of 


Factor Loadings Communality 
II III I+ lt I+ Tt Ttt-tv 
207 —.198 —.131 936 .992 
375 —167 —.022 172 .201 
—.387 —177 —.083 -905 943 
707 —155 —.192 .635 692 
—.089 112 —.012 .038 .051 
—.716 171 —.299 557 675 
443 167 152 897 .948 
—.255 —.360 272 429 .632 


entire test. These might be variable 1, gen- 
eral accuracy, or variable 3, recognition of 
statements not supported by the evidence; 
and either variable 4, true and false state- 
ments marked correctly, or variable 6, the 
tendency to be cautious in judging statements 
with less positivism than the data warrant. 
The third variable to be included if one were 
interested in shorter scoring is variable 2, 
general accuracy with probably true and 
probably false statements. 


CONCLUSIONS 


Internal evidence in this test indicates that 
it measures various skills in interpreting 
data. By definition there are five types of 
statements and the test gives an indication 
of the students’ skill in recognizing these 
statements. The test differentiates generally 
high from generally low ability and gives as 
well an indication as to the kind or type of 
ability. 

On the basis of the interrelations of the 
test variables, it may be concluded that 
possibly three of the variables reflect prac- 
tically all of the information given by the 
whole test. Considering the statistical find- 
ings, and the advisory use of the test data, 
these three variables which appear to be very 
important are: 


(3) Recognition of statements not sup- 
ported by the evidence. 

(6) Cautiousness—the tendency to be 
overcautious in judging the statements 
with less certainty than the data 
warrant. 

(2) Accuracy in judging the probably true 
and probably false statements. 
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III. COLLEGE STUDENTS AND 
CONTEMPORARY AFFAIRS 


O. O. ROYER 


Three basic assumptions are involved when 
a college utilizes contemporary affairs tests 
in its testing program. First, the college be- 
lieves that a knowledge of contemporary af- 
fairs is significant in the educative process of 
its students. The belief is general that the 
better educated person is one who knows a 
considerable amount about his contemporary 
culture, and the college shares that belief. 
Second, the school believes it can measure 
successfully the knowledge which students 
have of contemporary affairs. This factor in- 
volves the ability of the test makers to de- 
velop tests which are valid, reliable, prac- 
ticable, etc. It is quite conceivable that five 
different individuals could organize tests of 
current happenings on which each of the 
other five could fail to make a relatively high 
score. Third, the school believes that it can 
do something about the results of the tests 
in terms of pupil growth. No great purpose 
would be served for a school to learn how its 
students rated with respect to each other, or 
to other similar groups, unless that school 
also believes it can do something with its stu- 
dents to increase their knowledge of con- 
temporary affairs. Then it must also follow 
that the school will do something about the 
results of the tests. 

This study is concerned with the group of 
462 Education freshmen in terms of the three 
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basic assumptions which have been outlined 
It seems appropriate to discuss the second as. 
sumption first, namely, a school believes j 
can measure adequately its students knowl. 
edge of contemporary affairs. Table | 
presents a brief picture of the test which 
the College of Education used. Assuming 
that the test covers the major areas of cop. 
temporary affairs, it may be noted that each 
area is not represented by the same numbe- 
of items. Probably the last three areas «; 
Part I, with five, eight and twelve question: 
respectively, lack much in reliability. Th, 
low mean scores of these items relative : 
their standard deviations indicate the: 
distributions are skewed. 

There was no statistical study made with 
respect to the reliability and the validity 
this test as a measure of knowledge of cor. 
temporary affairs, but we shall assume in | 
rest of this discussion that the test 
reasonably reliable and_ valid. 

Table II indicates positive relationshy 
among all of the various areas in Part I. Thy 
knowledge of national personalities is particu. 
larly associated with knowledge of intern. 
tional personalities. An understanding of rea- 
sons for national events is closely associated 
with the students’ abilities to understand reo- 
sons for international events. There appear: 
to be a rather high degree of relations); 
between knowledge of personalities an‘ 
events, both national and international. One 
might have expected, however, that “Words 
in the News” would have had a higher rels- 


TABLE I 
MEANS AND STANDARD DEVIATIONS OF SUB-SCORES ON COOPERATIVE CONTEMPORARY AFFAIRS TPst 


Parts of Test 


TR Oi a 
Part I-—Political-Secial —.................... 


Identification of Personalities—National 


Identification of Personalities—International 


Knowledge of Events—National 


Knowledge of Events—International ____--~- 
Meaning of Words in the News _____------- 
0 a eS Seer 


Reasons for Events—National 
Reasons for Events—International 


Part II—Contemporary Culture _____.._______- 


ee ee 
aaa Sepciccnatieed 


Music and Radio _______-——— ss SsS 
| STA Rn, DT RE Be Sa 
A 


Ohio State 

Highest Education Freshmen. 

Possible Standard 

Score Mean Score Deviation 
ccapiaceamndmbisiial 300 71.36 30.22 
socket 150 29.80 16.87 
epriaeieestarte dis 22 8.02 3.39 
sactavinaeitatntt 18 7.12 2.89 
cvatuaapanioniichion 30 10.66 2.76 
otiaicinninppiamnts 40 12.51 6.02 
ideation 15 4.19 3.16 
See Se 5 -50 88 
sanepaaiasdiebiie s .95 1.83 
ncquianndbianten 12 1.11 1,18 
incising 150 41.09 18 24 
ee 50 14.27 5.68 
17 4.81 3.008 
aah 22 8.83 3. 
ws einateigubannibiie 43 25.81 8.83 
18 2.76 2.33 
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TABLE II 


INTERCORRELATIONS OF THE SUBDIVISIONS OF THE COOPERATIVE CONTEMPORARY 
AFFAIRS TEST (PART I) 


N = 462 


1. Total (Part I) Senate ee nae meen clean enn rR ENE ae NE TED Sn 


» Identification of National Personalities 


~> to 


4 Knowledge of National Events 


s. Meaning of Words in the News 
7, Science 
& Reasons for National Events 
9 Reasons for International Events 


tionship with other parts of the test. The 
comparatively low correlations may be due 
to an insufficient number of items in this part 
of the test. 

In Table III we see a close inter-relation- 
ship between the various tactors which make 
up the “Contemporary Culture” aspect of the 
whole test. The correlation between a know- 
ledge of books and of movies is significant 
by its being lower than the other correlations. 
\n important point is indicated for parents 
and educators. Some young people may be 
spending too much of their time with books. 
Part of that time might well be shared with 
other forms of amusement. On the other 
hand, many youth may be devoting too much 
time to the movies. These youth might well 
profit by more reading. 


Identification of International Personalities ___~ 


5 Knowledge of International Events _________ 


1 2 3 4 5 6 7 8 9 
73 .72 68 80 63 .25 44 .39 


es 60 52 0 384 .14 .25 .25 
72 .60 49 56 34 .17 19 115 
... 68 52 49 56 .25 .20 .20 .18 
_._ 80 .50 .56 .56 59 .28 .39 .39 
63 34 .84 .25 .59 40 51 AT 


25 .14 .17 .20 .28 .40 39 = .35 
44.25 19 20 39 51 .39 57 
a. OF 20 15 18 320 41 2 ST 
The first basic assumption discussed in this 
study stated that a school believes that a 
knowledge of contemporary affairs is signifi- 
cant in the educative process of its students. 
In considering this phase of our problem, we 
refer to Table IV which shows the relation- 
ship of the total scores of the Contemporary 
Affairs Test with rank in class, psychological 
test score, autumn point-hour-ratio, and Sur- 
vey of Education course grades. The rank in 
class refers to the students’ rank in their high 
school graduating classes, 2 indicating upper 
third, 1 middle third, and o lower third of 
the class. Form 19 of the Ohio State Psycho- 
logical Test was the basis for the second 
variable. The autumn point-hour-ratio was 
the third variable. This is the customary av- 
erage of University grades whereby four 


TABLE IIT 


INTERCORRELATIONS OF THE SUB-TESTS OF THE COOPERATIVE CONTEMPORARY 
AFFAIRS TEST (Part II) 


; Music & 

N = 462 Total Books Stage Radio Movies Art 
Ee eee 58 66 57 17 43 
SONI tossclasscssasancataldeldarinstnemnmen tii Gaitiniiniaideatbaniapiaiedi 58 55 45 .26 35 
SOUT winsnceiticielinidli ding thasiin tabiinihidejhibbaihishiceniacenonitionene 66 55 52 44 36 
on SE SRS SE Sa 57 45 52 38 31 
RRR ODE BERET ELE EA ORIIE 17 26 44 .38 38 
ES) ERIE IE EO SS ee CSS 43 .35 .36 31 38 

TABLE IV 


INTERCORRELATIONS OF TOTAL AND PART ToTAL ScoRES OF THE COOPERATIVE CONTEMPORARY 
AFFAIRS TEST AND FouR COMMON VARIABLES 


N = 462 Total PartI 
ef, ae ea EES 83 
oe | EEE ERTIES 83 
wn jh) aa 82 48 
Rank i | Se eae 18 14 
Psychological Test __...__________ 1 32 
Autumn P.H.R. tC 30 


Survey Grade 





Rankin Psych. Autumn Survey 


PartII Class Test P.H.R. Grade 
82 18 41 82 .27 
48 14 32 .30 .26 

19 42 .26 an 
19 45 44 37 
42 45 61 .50 
.26 44 61 .76 
.22 .37 .50 .76 
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points are awarded for each hour of A, three 
points for each hour of B, two points for each 
hour of C, and one point for each hour of D. 
The grade E (failure) receives no points. The 
Survey of Education grade was a grade for 
a particular course, especially designed for 
Education freshmen for purposes of helping 
them to adjust themselves to college work 
in terms of their chief interests and aptitudes. 

Now it would seem that one’s rank in a 
high school graduating class would indicate 
to a fair degree a particular student’s edu- 
cation in comparison with his fellow students. 
If our basic assumption is valid, one would 
expect a fair correlation between the rank 
in class and knowledge of contemporary af- 
fairs. The correlation of 0.18 does not bear 
out that expectation. Thus, if we hold to our 
basic assumption, one of two things seems to 
be true. Either the Contemporary Affairs 
Test is not valid, or the high schools use poor 
methods of measuring the amount of educa- 
tion which their students possess. A combina- 
tion of these two interpretations is also 
possible. 

The Ohio State Psychological Test was ad- 
ministered in the first week of school before 
the students were greatly influenced by uni- 
versity life. One is not surprised, however, 
at the high degree of relationship existing be- 
tween the Psychological Test and the Con- 
temporary Affairs Test as shown by a cor- 
relation of o.41. In brief, one can say that 
a group which does well on the Psychological 
Test tends to do well on the Contemporary 
Affairs Test. 

rhe correlation between the autumn point- 
hour-ratio and the Test total is 0.32. This 
would indicate that a higher degree of rela- 
tionship exists between college grading and 
knowledge of contemporary affairs than 
exists between high school grading and 
knowledge of contemporary affairs. 

Of the four outside variables studied, then, 
it is apparent that the Psychological Test 
shows the highest relationship with the total 
score of the Contemporary Affairs Test, and 
also with each of its two parts. Thus, one can 
very well say that the group which makes 
high scores on the Psychological Test also 
tends to have a fair degree of knowledge 
about contemporary affairs, both political- 
social and cultural in nature. 

The third basic assumption was that a 
school believes it can do something about in- 
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creasing its students’ knowledge of cop. 
temporary affairs. This involves more thap 
just presenting knowledge to be learned, fo; 
knowledge learned today will not be consi¢. 
ered contemporary knowledge one year from 
today. Thus, the problem at once become 
complicated and the school must of neces. 
sity develop in its students the desire tp 
acquire knowledge of contemporary culture 
as it is developing from day to day. A prac- 
tical question immediately arises. To what 
extent should a school expect its students to 
develop this desire for the acquirement oj 
contemporary knowledge? Since the desire 
itself means nothing unless it results in the 
actual acquiring of knowledge, the school has 
two things to do in order to establish a basis 
for the answer to the above question. First, 
it must secure an accurate picture of its own 
students’ accomplishments. Second, it should 
compare its group with other similar groups 
who have been measured with similar testing 
devices. 

In this connection, it is possible to com- 
pare the Ohio State freshmen with a great 
number of sophomores from sixty-six colleges 
where the same Contemporary Affairs Test 
was used.’ If we recognize the fact that a 
difference of at least one year in training 
existed between the two groups, we are ap- 
parently justified in believing that the Ohio 
State group did well in this comparison as is 
indicated by Table V. 

Another means of comparison, which also 
should be a measure of growth, will materia’. 
ize when the group of 462 return to the Ohi 
State University as sophomores, as it is the 
aim of the College of Education to again ac- 
minister to them a Contemporary Afiais 
Test. 

If as sophomores, the Ohio State group 
make scores similar to a general college group 
of sophomores, another point should be 
clarified before an interpretation is attempte¢ 
If the colleges in the general group are mak- 
ing definite attempts to assist their students 
to acquire knowledge of their contemporar) 
lives, then the Ohio State University College 
of Education may have the right to a feeling 
of satisfaction. If, however, the figures [or 
the general group would be the result of the 
promiscuous use of tests by the colleges a 
the group, the College of Education may wel 
be disappointed with its efforts. With ™ 


1 Eckert, Ruth E. “Who Are the ‘Cultured’ in Our Col- 
leges?” The Educational Record, January, 1939. 
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TABLE V 


COMPARISON OF SOPHOMORES OF SIXTY-SIx COLLEGES WITH THE OHIO STATE 
UNIVERSITY EDUCATION FRESHMEN 


Ohio State Freshmen 


Part I 
Mean Score -------------------------- 29.8 
Standard Deviation ~------------------ 16.9 
Nomber -.-~------.- +... we 462 


training, the Ohio State group should measure 
up to a general group. With training, they 
should exceed the general group. 


SUMMARY 


For a college or university to administer 
intelligently a contemporary affairs test as a 
part of its testing program, it must accept 
three basic assumptions. First, the ade- 
quately educated individual is one who knows 
considerable about contemporary affairs. Sec- 
ond. the school believes it can measure its 
students’ knowledge of contemporary affairs. 
lhird, the college or university believes it can 
influence its students in such a way that they 
will acquire knowledge of contemporary 
affairs as the new knowledge develops. 

The Ohio State University administered 
the Cooperative Contemporary Affairs Test 
to its Education freshmen in the Autumn 
Quarter of 1938. By the very act of ad- 
ministering this test, the University gave evi- 
dence of accepting the first assumption. The 
results indicate, however, that knowledge of 
contemporary affairs is not closely associated 
with other factors which are commonly indi- 
cative of student education, namely, grades 
and rank in class. It is of interest to note 
that there is a high degree of relationship be- 
tween the test results and the Psychological 
Test, which indicates that those with superior 
ability tend to know more about their sur- 
rounding culture than those of lesser ability. 

By giving the Contemporary Affairs Test, 
the College of Education indicated its ac- 
ceptance of the second basic assumption. 
However, the items in some parts of the test 
are so few, the mean scores are so low, and 
the intercorrelations are of such a degree that 
one immediately is assailed by serious doubts 
about the validity of the test which was used. 
Also, there is little evidence that the test is 
reliable. 

In formulating the purposes of the Survey 
course, the College of Education also gave 
evidence that it accepted the third basic as- 


College Sophomores 


PartII Total Part I PartII Total 
41.1 71.4 38.4 45.4 83.7 
18.2 30.2 22.7 21.7 39.3 
462 462 6378 6378 6378 


sumption. However, no evidence has yet been 
presented that this group of 462 students im- 
proved in their knowledge of contemporary 
affairs, as no comparable test was given at 
the end of the year. Further testing is a much 
needed next step. Definite procedures should 
be established not only for the teaching of 
a knowledge of contemporary affairs, but also 
for the furthering of the desire in the indi- 
vidual to master new knowledge as it appears. 


IV. LIBERALISM AND CONSISTENCY: 
A STUDY OF SOCIAL ATTITUDES 


C. C. Gresons AND W. A. B. SCHRADER 


The results obtained by giving a test called 
“A Scale of Beliefs,’ to a group of education 
freshmen constitute the data for this study. 


The test consists of one hundred eighty-six 
statements which deal with the following 
types of social problems: 


(1) Democracy 

(2) Economic Relations 
(3) Labor 

(4) Race 

(5) Nationalism 

(6) Militarism 


The students were toid that “there are no 
right or wrong answers; you are to express 
your point of view about these statements.” 
The students indicated their attitude toward 
each statement in the test by marking them 
as either “Agree”, “Disagree”, or “Uncer- 
tain”. 

The following statements are indicative of 
the types of statements included in the test: 


1. In a democracy, it is often necessary to 
give up one’s own desires or to change 
one’s actions so as to benefit others. 
(Democracy) 


1“A& Scale of Beliefs’, Progressive Education Association 
Tests 4.2 and 4.3, Evaluation in the Eight-Year Study. 
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2. Our economic progress would be greater 
and our standard of living higher if our 
industries were publicly owned and op- 
erated. (Economic) 

3. The government should provide for the 
unemployed through taxation. (Labor) 

4. All positions in the political and eco- 
nomic world should be open to any man 
with the ability to fill them, regardless 
of race. (Race) 

5. Other countries excel us in many 
respects, just as we excel them in many 
respects. (Nationalism) 

6. A man who is sincerely against war 
should not be made to go to war. 
( Militarism) 


If a student agrees with one of the fore- 
going statements, he receives one score to- 
ward liberalism in the area indicated. If, on 
the other hand, he disagrees with one of these 
statements, his response is counted as “con- 
servative’’. In this way, the test papers were 
scored for liberalism and conservatism in 
each of the six areas mentioned above. The 
number of items which a person marks “un- 
certain” is taken as his uncertainty score. One 
week after the students had completed one 
part of the test, which consists of ninety-three 
items, they were given a second part which 
contains items similar to those in the first 
part. Each statement in part two expresses a 
viewpoint directly opposed to the viewpoint 
expressed by the corresponding statement in 
part one. By considering the answers to these 
pairs of statements a consistency score was 
derived for each of the areas of thought in- 
volved. The six statements from part two 
which correspond to the six statements listed 
above from part one are: 


1. In a democracy, everyone should be 
free to do as he pleases and should not 
be forced by the government to do 
things he does not wish to do. 
( Democracy ) 

2. Production based on the system of pri- 
vate ownership is essential for economic 
progress. (Economic relations) 

3. The government has no right to tax the 
public generally to provide for those 
who are out of work. (Labor) 

4. Negroes should not be allowed to fill 
positions involving leadership over white 
people. (Race) 
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5. The United States is superior to mos 
other nations in such important respects 
as government, educational opportuni. 
ties, family life, and morals. (Nationa). 
ism) 

6. No matter what his personal beliefs re. 
garding war, no man has the right +, 
refuse to serve his country in times oj 
war. (Militarism) 


It is apparent in most cases that a persop 
who agrees with a certain statement in part 
one will, if he is consistent, disagree with the 
corresponding statement in part two. Some 
of the statements in the first ninety-three are 
so stated that the “liberal” will agree with 
them, and some are so stated that the 
“liberal” will disagree with them. 

The test yields twenty-eight scores ev. 
pressed in percentages as follows: Scores for 
liberalism, conservatism, uncertainty, and 
consistency, for each of the six areas tested, 
and total scores for liberalism, conservatism. 
uncertainty and consistency. In addition + 
these twenty-eight test scores, sex, race, in- 
telligence test score, and college grades are 
included in this study. 

Standard Hollerith equipment and methods 
were used throughout this study in the 
computation of the means, standard devia- 
tions, and coefficients of correlation. The 
original population included both negro and 
white, male and female students. Means and 
standard deviations were computed for the 
four race-sex groupings, in each of the vari- 
ables. At this point the negroes were elimi- 
nated in order to obtain a more homogeneous 
population for the calculation of intercorrela- 
tions. The negroes were not studied sepa- 
rately because the size (N = 25) and rep- 
resentativeness of the sample did not seem 
to justify such an analysis. It may be noted, 
as indicative of the validity of the test, that 
the negroes were decidedly more “liberal 
and more consistent than the whites in at 
titude toward race problems. For the remain- 
ing four hundred thirty-seven subjects, 
means, standard deviations, and correlation 
coefficients were obtained for the following 
variables: 


Variable number— 
1. Scholastic Grades? 


2. Intelligence 


? The grades used were point-hour ratios for autumn quarter 
1938 





sf 


Liberalism in: 
;. Democracy 
4. Economic Relations 
5. Labor 
Race 
-. Nationalism 
8. Militarism 


Consistency in: 
Democracy 
:o. Economic Relations 
11. Labor 
12. Race 
_ Nationalism 
Militarism 
fotal Liberalism 
Total Conservatism 
17. Total Uncertainty 
i$. Total Consistency 


TA 
bo 
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Unfortunately, these tests were scored in 
such a way that it was not feasible to cal- 
culate coefficients of reliability for the test. 
The Progressive Education Association, pro- 
ducers of the tests, report reliabilit, coeffi- 
cients which are based on 500 subjects, rang- 
ing from grades 9 to 12. Reliability coeffi- 
cients were estimated from the data of the 
present study by the method of Kuder and 
Richardson.* These two sets of reliability 
coefficients are presented in Table II. 

The relation of the several liberalism 
scores to intelligence and college grades is 
shown in Table III. 

Comparisons between means for different 
kinds of liberalism would not be valid be- 


3G. F. Kuder and M. W. Richardson, “The Theory of the 
Estimation of Test Reliability’, Psychometrika II: 151-160, 
September, 1937. 


TABLE I 
MeANs, STANDARD DEVIATIONS, AND CORRELATIONS FOR LIBERALISM VARIABLES 
Stand- 
Ec. ard De- 
Demoe. Rel. Labor Race Nation. Milit. Total Mean viation 
ic} Ee 53 1 06 ol 48 .76 65.6 10.0 
M 46 52 30 4 46 58 61.9 9.1 
nic Relations __--_ F .o3 8 a oO 40 70 52.2 18.4 
M 46 5S 20 85 83 64 49.5 18.1 
spas aeobaks sae ol 48 43 40 ot oti 65.7 14.7 
M452 55 20 36 33 60 63.6 14.2 
pagekieibn: ee 36 .22 43 46 26 56 59.1 23.1 
M30 20 320 8 33 58 58.5 21.3 
SUN octane F ol 38 .40 46 54 .70 61.6 15.7 
M .40 35 36 3 56 62 56.4 15.6 
ree F 48 40 37 .26 4 67 62.5 13.7 
M 4 33 33 33 6 61 57.5 14.0 
ail cca F .76 .70 71 6 70 67 60.2 10.9 
M 08 64 .60 58 62 61 57.7 10.4 
LIBERALISM TABLE III 
the correlations between liberalism in = CorReLATIONS BETWEEN LIBERALISM, INTEL- 
cifferent areas, computed separately for men LIGENCE, AND COLLEGE GRADES 
and — are shown in Table I. For all Liberalism in: Correlation with: 
the tables, the figures for men are in italics Intelligence Grades 
ind those for women are in arabic type. The (3) Democracy ____ F 30 234 
ngures are also labelled “M” and “F’’. M 25 06 
(4) Economic Rela- 
TABLE II OE ecco F 14 .20 
RELIABILITY COEFFICIENTS FOR LIBERALISM M 08 06 
VARIABLES (5) Leber ........ F .29 25 
Liberalism in: P.E.A. K-R Method M 12 07 
F M ff F 27 24 
c emer acy en 80 58 46 M 18 08 
{ y ; $ 75 74 io 7 : 
(5) ee Relations = 71 63 (7) Nationalism ___ F 30 31 
AO OE ndiicsieessiacion 87 81 9.77 = - ad 
(7) Nationalism _____ .70 63 60 (8) Militarism __.__ F 3 36 
(8) Militarism _..____ . 82 71 71 M 28 27 
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cause they would reflect nothing more than 
the difficulties of the different tests. Sex 
differences in means, however, may be legit- 
imately compared. We shall consider first the 
question of sex difference in means of the 
liberalism variables. The only differences 
that are presented in Table IV are those that 
are at least twice their standard errors. For 
every difference included in Table IV the 
mean for the women was larger. 


TABLE IV 


CRITICAL RATIOS FOR SEX DIFFERENCES 
IN MEANS 


Name of Critical Ratio 
Variable for Sexes 
Intelligence 3.92 
Liberalism in Democracy ~--~-~- 3.93 
Liberalism in Nationalism a 3.29 
Liberalism in Militarism oe 3.39 


For determining the significance of sex 
differences in the intercorrelations, these cor- 
relations were changed into Fisher’s “z” 
function.* The correlations for which there 
are significant sex differences are those be- 
tween liberalism in democracy and (1) total 
liberalism (3.10), and (2) college grades 
(2.91). These numbers are the critical ratios 
which indicate the extent to which the cor- 
relations were higher for women. 

In addition to the sex differences, the dif- 
ferences between the magnitudes of certain 
correlations are interesting. For instance, 
Table III shows that for men the correlation 
between intelligence and _ liberalism in 
economic relations is 0.08 whereas the cor- 
relation between intelligence and liberalism in 
nationalism is 0.35. How much emphasis 
should be placed on such a difference? This 
paper presents one hundred twenty-nine in- 
tercorrelations for women and the same num- 
ber for men. It would be possible to make 
8256 comparisons like the one above for men 
and the same number for women. From 
Tables V and VI the reader can estimate the 
significance of the difference between any 
correlations which he wishes to compare in 
this study. Tables V and VI show for the 
two groups in this study the critical ratios of 
the differences in every case where the 
critical ratio is larger than 2.00. 

Table IV shows that one of the largest sex 
differences in means is in intelligence. As ex- 


* Fisher, R. A. Statistical Methods for Research Workers. 
Oliver and Boyd, London 1938 z % [log, (1 + r) — log, 


(1—r)). 
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perience has shown to be true of educati, 
students, the women in this study have - 
higher mean intelligence test score than + 
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men. The question immediately arises , 


TABLE V 


CRITICAL RATIOS OF THE DIFFERENCES Betws: 


r, AND r, WHEN N = 


J 288 (E.G., 
FEMALE POPULATION IN THIS Stupy) 


Yr: 
30 86.40 = .50 
10 2.53 3.95 5.48 
.20 2.67 4.23 
Yr: .00 2.86 
40 
50 


TABLE VI 


CRITICAL RATIOS OF THE DIFFERENCES BeTWrr 


r: AND r. WHEN N = 


MALE POPULATION IN Tus Stupy) 


.40 
10 2.83 
.20 
ri 30 
40 
50 


whether this difference may not explain th 
other sex differences presented in Table I\ 
The correlations in Table IIT between int: 
ligence and different kinds of liberalism av 
so low, however, as to indicate that int 
ligence does not bear enough relationship 
liberalism to cause any marked sex differ 


ences. 


Table I shows considerable interrelation be- 
liberalism. Hoy 


much of this interrelationship is due to th 
Table VI 


tween different kinds of 


149 (E.G., 
r: 

.50 60 
3.92 5.21 
3.03 4.26 
2.08 3.30 


common factor of intelligence? 


shows the correlations of Table I after th 
linear influence of intelligence has been re 


moved by partial correlation. 


It is noteworthy that the correlations » 
Table VII are still significantly greater than 
zero after intelligence has been held constan' 
The largest reduction caused by partialling 
out intelligence was 0.08 for women and 0. 
for men. The different liberalism scores see" 
to have something in common besides inte 
ligence. It must be recognized that the ony 
liberal behavior involved here is in making é 
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certain response to a written statement. 


Can the intercorrelations between different 
kinds of liberalism be taken as an indication 
of a liberalism factor? Part of the arg 





THI 
Lilt 
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TABLE VII 


Democ. 


ee a See ee ee F 
t cracy M 
c Relations — 2 
M 46 
F AT 
M 51 
F } 
M 27 
\ ‘i aan = AT 
M sh 
M al F 41 
M 42 
here must be based on Table VIII, 
shows correlations between different 


liberalism scores and the corresponding con- 
sistency scores. Table I shows correlations 
hetween different kinds of liberalism. The 
eans’ of these correlations in Table I are 
423 for women and 0.388 for men. One 
cht regard these figures as evidence for a 
beralism factor. However, these means are 
t little larger than the means of the cor- 
relations in Table VIII between different 
nds of liberalisms and the corresponding 
nsistencies. The means in the latter case 
ve 0.334 for women and 0.321 for men. 
[he correlations on the diagonal in the 
wer part of Table VIII are noticeably high; 
e, liberalism in race with consistency in 
ce, etc. 
eans were computed before the correlations were 
two decimal places. 


TABLE 


RRELATIONS BETWEEN LIBERALISM VARIABLES WITH INTELLIGENCE HELD CONSTANT 
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Ec. Rel. Labor Race Nation. Milit. 
52 AT 31 AT Al 
46 ol 27 4 42 

47 19 36 38 

6 19 #5 82 
AT 3 4 29 
5 28 4 31 
19 38 42 18 
19 28 och 20 
36 04 42 A8 
Jd Jt 34 mF | 
38 .29 18 48 
PA 4 my 20 RF i 


An interesting sex difference occurs in that 
the correlation between liberalism in race and 
consistency in race is larger for women than 
for men. The critical ratio is 2.09. A differ- 
ence that is hard to explain occurs in the 
correlation between total liberalism and con- 
sistency in attitude toward labor. The cor- 
relation for men is higher to the extent of a 
critical ratio of 2.38. 


CONSISTENCY 


On the first form of the test, ninety-three 
statements are presented, some worded so 
that a liberal answer is ‘Agree’ and some 
worded so that a liberal answer is ‘Disagree’. 
On the second form of the test, each state- 
ment is reworded so that it expresses the op- 
posite point of view. Consistency is measured 
by the extent to which the person makes the 


CORRELATIONS BETWEEN LIBERALISM AND CONSISTENCY SCORES 


steney in: Democ. Ec. Rel. 
ntelligence i .27 .24 
M 27 35 
ee F .34 24 
M 19 34 
alism in: 
Ds F 59 .20 
M 55 27 
Economic Relations _. F 36 Jl 
M 26 30 
NE oe ee 39 .22 
: M 44 16 
Race Se ep ee F .29 14 
M "25 26 
Nationalism _..______ F 41 .23 
M 43 41 
Militarism _...______ F 04 .29 
M 25 36 
Total Liberalism _____ F 53 .30 
M 46 44 





VIII 
Total 
Labor Race Nation. Milit. Consist. 
ol 20 .29 36 ie 13) 
24 13 28 26 0 
oo 21 3 35) 40 44 
.20 08 382 24 35 
04 2 00 36 AT 
44 A? 35 23 A4 
06 13 .33 ol 38 
A3 17 22 25 ST 
51 25 oo wl .46 
58 03 26 19 389 
.28 62 25 .20 42 
89 47 25 2 42 
34 .26 59 41 52 
87 24 46 40 52 
27 19 43 .65 45 
39 20 .26 55 47 
44 38 .48 51 59 
-61 33 389 51 61 
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TABLE IX 


MEANS, STANDARD DEVIATIONS, 
Ec. 
Democ. Rel. Labor 
Democracy I wv 45 
M 38 19 
Economic Relations. F 0 33 
M 38 41 
Lal ’ Kk 15 3 
M 49 41 
Rac Fr .24 22 .28 
M 19 26 8 
Nationalisn F 6 aa ot 
M 18 B 6 
Militarisr I 15 27 .08 
M 32 43 42 
lo F 68 0 .63 
M 66 70 69 


same types of responses (i. e., either liberal 
or conservative) on the corresponding state- 
ments of the two test forms. Table IX shows 
the correlations between different consistency 
scores. 

The reliability coefficients of the consist- 
ency scales were not directly obtained. The 
reliabilities were estimated by the Kuder— 
Richardson method referred to above. The 
Progressive Education Association reports re- 
liability coefficients for these scales, based on 
500 subjects in grades 9 to 12. Both sets of 
reliability coefficients are presented in 
Table X. 

The sex differences shown in Table IX 
were tested for significance. The variables for 
which the differences in means are significant 


TABLE X 


RELIABILITY COEFFICIENTS FOR CONSIST- 
ENCY VARIABLES 


Seale Coefficient of Reliability 
Consistency in: P.E.A. K—R Method 
F M 

(9) Democracy ___.--~. .68 41 39 
(10) Economie Relations .88 2d 40 
a Sees 61 A6 39 
(aa 51 46 
(13) Nationalism ___-~. .66 .39 25 
(14) Militarism scubic sic ahicte, .56 45 


are consistency in democracy (2.93) and 
consistency in economic relations (2.03). 
These figures are the critical ratios. Both 
means are larger for the women than for the 
men. To facilitate their comparison the cor- 
relations were changed to Fisher’s “‘z’’ func- 
tion. The only correlation for which ‘en is 
a significant sex difference is that between 


AND CORRELATIONS FOR CONSISTENCY 


VARIABLES 


Ny 


“landa 
Mean Deviatio 


Race Nation. Milit. Total 
24 36 45 .68 60.2 12 
19 38 2 66 56.5 12 
22 .22 27 50 59.4 16. 
26 7 43 70 55.8 17.4 
.28 37 .38 63 63.6 15.5 
33 36 42 69 62.6 Ii 
24 19 AD 61.2 | 
15 Js 49 57.5 4 
24 46 04 56.3 178 
15 43 9 55.4 7 
19 46 64 60.1 1f 
ms o/ 43 .67 57.8 , 
49 54 64 59.4 ) 
49 59 67 56.4 


consistency in economic relations and totg 
consistency. The correlation for men 
higher to the extent of a critical ratio of 3.15 
This is probably an underestimate becaus 
of the correlation of the two variables. 


Table [IX shows some correlation betweer 
consistencies in different areas of the test, T 
determine how much of this correlation was 
due to the common factor of intelligence, 
Table XI was calculated to show the relation 
between different consistency scores when in- 
telligence is held constant. The correlations 
in Table XI are still significantly greater than 
zero after intelligence has been held 
stant. The largest reduction resulting from 
partialling out the intelligence test score ‘s 
0.08 for women and 0.07 for men. 

The mean of all the correlations in Tab 
IX between different consistency scores is 
0.317 for women and 0.349 for men." These 
figures probably cannot be taken as evidence 
for a general consistency factor, however. be- 
cause the means of the correlations between 
liberalism scores and consistency scores are 
0.334 for women and 0.321 for men. (%¥ 
Table VITT) 


CONSERVATISM AND UNCERTAINT\ 


As explained at the beginning, there wer 
conservatism and uncertainty scores in ea 
of the six areas as well as liberalism and co 
sistency scores. This study has been limite: 
chiefly to a consideration of liberalism an 
consistency. Total conservatism and uncer 
tainty scores were studied, however, in rela- 
tion to intelligence, grades, and total liber’ 

*See footnote 5. 
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TABLE XI 
ConRELATIONS BETWEEN DIFFERENT CONSISTENCY ScoRES WITH INTELLIGENCE HELD CONSTANT 
Democ. Ec. Rel. Labor Race Nation. Milit. 
re ar F .26 40 19 30 39 
‘ : M 32 45 16 33 27 
Economie Relations ------------- F 26 27 18 16 20 
M 2 36 23 30 37 
iii nee Satnienetanapilcieii at F 40 27 .22 31 30 
M 45 36 P 31 38 
ili cnciciesse aenetbiasdetaenanin anagem F 19 18 .22 19 12 
M 16 23 a 11 S82 
Nationalism ome eww ow ee eo wee oe ee = F .30 16 ol 19 39 
‘ M 33 30 31 11 39 
I csiiksiieme snememenipiasininaatetin F 39 .20 30 12 39 
M 27 S87 38 32 39 
TABLE XII 
MEANS, STANDARD DEVIATIONS, AND CORRELATIONS FOR INTELLIGENCE, 
GRADES, AND TOTAL SCORES 
Total Total Total Total 
Liber- Conserv- Uncer- Consist- Standard 
Intell. Grades alism atism _ tainty ency Mean Deviation 
Intelligence _------ F 60 37 —.21 —.13 35 65.1 24.8 
M 8 £1 —.40 06 9 55.0 26.1 
pn ena eS F 60 oT —.27 —.13 44 2.39 15 
M 8 ae —.29 82 35 2.28 -67 
Total Liberalism -.. F wT OT —.27 —.69 59 60.2 10.9 
M £1 17 —.30 —.50 -61 57.7 10.4 
Total Conservatism. F —.21 —27 —.27 —12 —.35 20.5 9.6 
M —.40 —.29 —.30 — 33 —.28 22.5 9.4 
Total Uncertainty... F —.13 —13 —.69 —.12 —.27 20.0 10.5 
M 06 82 —.50 —.33 —.29 21.1 13.1 
Total Consistency __ F 35 44 59 —.35 —.27 59.4 10.9 
M 39 35 -61 —.28 —.29 56.4 10.7 


sm and consistency scores. These results are 
presented in Table XII. 


In Table XIII, the coefficients of reliability 
which the Progressive Education Association 
present for the total scores are compared with 
those estimated by the Kuder—Richardson 
method. 

Some of the sex differences presented in 
Table XII are significant. For total liberal- 
sm and total consistency, the women have 
nigher scores to the extent of critical ratios of 
230 and 2.77 respectively. For total con- 
vervatism and total uncertainty the differ- 
ences are in favor of the men, the critical 
ratios being 2.09 for conservatism and 0.91 
for uncertainty. As indicated previously, the 
women scored higher on the intelligence test. 
The critical ratio of this difference is 3.82. 

A comparison of the correlations by Fish- 
ers “2” function shows that the only signifi- 
cant sex differences in Table XII result from 
the correlation of total uncertainty with: 


grades (4.45), total liberalism (2.89) and to- 
tal conservatism (2.21). These numbers are 
the critical ratios. 


SUMMARY 


The Education freshmen at Ohio State uni- 
versity were given an attitude test to deter- 
mine their reactions to statements in the fields 
of democracy, economic relations, labor, 
race, nationalism and _ militarism. These 
tests were scored for liberalism, conservatism, 
uncertainty and consistency in each of the 
six fields. Intelligence test score and first 


TABLE XIII 


RELIABILITY COEFFICIENTS FOR THE 
TOTAL SCORES 


P.E.A. K-R Method 
F M 
Total Liberalism ______ .92 .90 89 
Total Conservatism —___ .90 91 90 
Total Uncertainty _.._ .88 93 96 
Total Consistency -___._ .96 80 79 
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TABLE XIV 
MEANS OF DIFFERENT TYPES OF 
INTERCORRELATIONS 
Women Men 
15 r’s for Liberalism vs. Liber- 
alism 4233 7885 
6 r’s for Liberalism vs. Con- 
sinteney 3336 2209 
15 r’s for Consistency vs. Con- 
sistency 173 3489 


quarter college grades were also included in 
the study. The means, standard deviations 
and intercorrelations were obtained for these 
variables for a group of four hundred thirty- 
seven men and women. All the sex differences 
found were tested for significance and the 
differences which are twice their standard 
errors are reported. For intelligence, grades, 
and the fourteen liberalism and consistency 
variables the means were all higher for women 
than for men. Eight of the critical ratios were 
above 2.00. Only for total conservatism and 
total uncertainty do the men have a higher 
mean. Of the one hundred twenty-nine dif- 
ferences in correlation, eight of them were 
more than twice their standard errors. 

The intercorrelations between different 
liberalism scores were significantly greater 
than zero. These correlations were not sub- 
stantially reduced when intelligence was held 
constant. The same may be said for the inter- 
correlations between different consistency 
scores. These intercorrelations raise the ques- 
tion as to whether there may not be a general 
liberalism and a general consistency factor. 
Such a contention is weakened, however, by 
the data of Table XIV which shows the 
means of the different types of intercorrela- 
tions. There seems to be a stronger argument 
for a liberalism factor than for a consistency 
factor. If the figures in Table XIV_ had 
been based on correlation coefficients that had 
been corrected for attenuation, the case for 
generality of the traits would look stronger. 
Perhaps the intercorrelations are large enough 
to justify one to speak of liberalism in A, 
liberalism in B, etc. If the traits were un- 
correlated, one would not be justified in using 
the term “liberalism” in every case. The case 
for general liberalism is not strong. Taking 
.423 from Table XIV as the average inter- 
correlation between different liberalism 
scores, one arrives at the interpretation that 
there are only 59 chances in roo that a per- 
son above average on one kind of liberalism 
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will be above average on a second kind 
liberalism, while chance alone would give ;- 
chances in 100. 

Tables are given from which one may est). 
mate the critical ratio of the differe: 
tween two correlations which he wishes ; 
compare. Besides: the intercorrelations by. 
tween liberalism scores and 
scores in different areas of thought, the jp. 
tercorrelations are given between intelligence 
grades, total liberalism, total conservatisy 
total uncertainty and total consistency scores 

It is important to notice how low the « 
sistency scores are in this study. The 
dent was presented with two statements dur 
ing the course of the test. If he answered | 
statements liberally, or conservatively, he wa: 
given credit for being consistent. The mea 
scores show that the responses were consistent 
less than sixty per cent of the time. hy 
chance alone the responses would have bee: 
consistent thirty-three percent of the time 
A mean score of sixty per cent may be inter- 
preted to mean: (1) The students gay 
thought to their responses; hence this tes 
is invalid; (2) The students fail to recognizx 
that the second statement in the pair is 
direct conflict with the first; (3) The low 
consistency scores indicate that freshmen ar 
confused about their opinions on_ social 
issues. There are several bits of evidence 
which support this third interpretation. The 
consistently positive correlations presented 
this study would have been highly improba 
if the students had simply guessed. High 
consistency scores are associated with low 
uncertainty scores. There seems to be some 
relationship between the amount of experienc’ 
and discussion in a certain field and the con 
sistency with which college freshmen answer 
questions in this field. Following are the s\ 
fields involved in this study ranked in the 
order of their mean consistency score: 


Consistency 


Women Men 
1. Labor 1. Labor 
2. Race 2. Militarism 
3. Democracy 3. Race 
4. Militarism 4. Democracy 
5. Economic Rela- 5. Economic Re! 
tions tions 
6. Nationalism 6. Nationalism 


The ranking is roughly the same when th 
fields are arranged in order of mean liberal 
ism score. Are not the fields here ranked 4° 
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cording to the amount of experience and 
discussion in which the average college fresh- 
»an has participated in connection with the 
fields in question? 


CONCLUSIONS 


lhe women included in this study are 
~ore liberal, more consistent and more cer- 
sin of their beliefs than the men. These dif- 
ferences are presumably due to cultural 
rather than biological factors. The fact that 
the women of this sample have higher intel- 
ligence test scores indicates that a small part 
{this apparent sex difference may be due to 
the difference in intelligence. Several other 
studies have shown that there is a positive 

rrelation between liberalism and _intel- 
igence.. Murphy attempts to explain this 
relation by saying that more intelligent 
persons are bombarded by a different set of 
cultural influences than the less intelligent. It 
is also possible that the more intelligent have 
more test-taking skill. 

There is a much larger negative correla- 
tion between uncertainty and liberalism than 
between uncertainty and conservatism. It 
may be that those who hold liberal views de- 
velop an emotional conviction to support their 
opinions. The views of the conservative are 
usually traditional and do not need to be so 
staunchly defended. Another possible ex- 
planation is that the test measures liberal- 
sm more accurately than it measures 
conservatism. 

The correlations between liberalism in 
race and other kinds of liberalism are lower 
than for any comparable set of correlations. 
This seems to indicate that college freshmen 
have “blind spots” in the field of race more 
frequently than in other fields of thought. 
The correlations in the field of democracy 
are highest, possibly because “democracy” is 
here used as a rather inclusive concept. 


4. There is a rather high correlation be- 
tween liberalism in a certain field of thought 
and consistency in that same field. 

5. Consistency of response bears a positive 
relationship to liberalism. It is also positively 
related to intelligence, but less closely than to 
liberalism. 


. @ &, Murphy, & T. Newcomb, Experimental Social 
fy, p. 930, New York, Harper, 1937, xi + 1121 
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V. ANALYSIS OF VARIANCE APPLIED 
TO LIBERALISM SCORES 


W. A. B. SCHRADER 


The problem as to whether or not a sig- 
nificant relationship exists between two vari- 
ables, one of them quantitatively measured 
and the other only verbally indexed, is one 
which frequently arises in psychology and 
education. This study may serve as an il- 
lustration of the usefulness of analysis of 
variance in attacking such problems. 


In this study, the variable which is quan- 
titatively measured is the total liberalism 
score on the Progressive Education Associa- 
tion test called “A Scale of Beliefs’. This test 
has been described in detail in Section IV, to 
which reference may be made for informa- 
tion about the particular test used. The 
characteristics of the subjects for which less 
refined indications are available are: (1) 
Best liked high school subject; (2) Mother’s 
education; (3) Father’s education; and (4) 
Father’s occupation. Of these, the first was 
recorded by the subjects during freshman 
week in connection with the intelligence test- 
ing program, and the other three were in- 
cluded in their application blanks, which 
they filled in before being admitted to the 
University. 

The population included 437 white fresh- 
man students (288 women and 149 men) in 
the College of Education of the Ohio State 
University. These freshmen took the ‘Scale 
of Beliefs” test in connection with a survey 
course in education. 


It can be argued that the best liked high 
school subjects should be related to liberal- 
ism. For example, students who liked social 
studies best would be expected to be, on the 
whole, more liberal than students who pre- 
ferred mathematics. The approach here, 
however, is to examine the question by a 
statistical rather than a logical analysis. To 
deal with this issue, we first group the stu- 
dents according to best liked high school 
subject, and compute a mean liberalism score 
for each group. These means will in general 
differ from the mean for the entire sample, 
even if there is no relationship between the 
basis for classification and the dependent 
variable; but if there is a relationship, the 
variation in means among the groups should 
be larger than would be expected from 
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chance. By the use of the F-test’ it is 
possible to determine the likelihood that a 
given set of differences between the group 
means and the mean of the total group would 
have arisen by chance. Evidence for a rela- 
tionship consists of an F—ratio for the data 
so large that its occurrence due to chance 
alone would be very improbable. Stated in 
other terms, the null hypothesis, which in 
this case would be that no relationship exists 
between best liked high school subject and 
liberalism, is considered to be disproved if 
the value of F turns out to be larger than 
would occur by chance in one per cent of 
experiments. 

The value of F is determined from the data 
by dividing the “mean square between 
means” by the “mean square within groups.” 
In this ratio, the numerator measures the 
variance attributable to differences between 
the various groups and the denominator meas- 
ures the variance which is to be attributed to 
other factors than the one used in making 
the classification. 

In this study, the following grouping of 
best liked high school subject was used: so- 
cial studies, mathematics, biology, physical 
science, foreign language, English, vocational 
arts, fine arts, physical education, and com- 
mercial studies. Table I presents the results 
of the analysis of variance for this classifica- 
tion. For both sexes, the mean square be- 
tween groups is smaller than the mean square 
within groups, which tells us immediately that 

! Snedecor, G. W. Statistical Methods. Collegiate Press, Inc., 


Ames, lowa. 1937 
TABLE I 


ANALYSIS OF VARIANCE: LIBERALISM VS. BEST 
LIKED HIGH SCHOOL SUBJECT 


Women* 
Sumof Degreesof Mean 





Squares Freedom Square 
Between groups... 3.83 9 43 
Within groups____ 336.59 277 1.22 
Total ___._.. 340.42 286 
Men 
Sumof Degrees of Mean 
Squares Freedom Square 
Between groups... 5.33 9 59 
Within groups____ 157.21 139 1.13 
Total ....... 162.54 148 


° Omitting the one no-data case on best liked 
subject. 
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such a situation could easily arise by chance 
so that no F was computed. An examinatioy 
of the means of the groups does not lead ; 
any hypothesis for combining groups, by; 
rather supports the position that liberalis» 
and best liked high school subject as meas. 
ured in this study are not related. 

Tables II, III, and IV present the meay: 
and analysis of variance for mother’s edyca. 
tion in relation to liberalism. It wil! be ob. 
served that although the value of F js po 
significant when mother’s education is divided 
into six categories for either sex of student: 
a highly significant value arises in the cay 
of the men when the grouping is into on) 


TABLE II 


MEANS* AND NUMBERS OF CASES: LIBERALIS\! 
ScoRE ACCORDING TO MOTHER’sS EpDUCATION 











Mother Students 
Women Men 
N M . M 
1. College graduate ______ 15 5.3 7 6. 
2. Attended college but did 
not graduate _______ 59 5.6 20 58 
3. High school graduate__ 98 5.4 41 52 
4. Attended high school but 
did not graduate ____ 46 5.4 24 5 
5. 8th grade or less _____-~ 57 5.8 45 52 
. eee 13 5.9 12 5 
Te 288 5.6 149 is 
*In this study, scores were coded according 
to the formula X = 10X’ + 4.5. The mean 
and variances reported are based on the tra 
muted (X’) scores. 
TABLE III 
ANALYSIS OF VARIANCE: LIBERALISM SCoRE 
ACCORDING TO MOTHER’S EDUCATION 
Women 
Degrees 
Sum of of Mean _ 
Squares Freedom Square 
Between groups 8.03 5 161 
Within groups_ 332.72 282 1.18 
Total ____-- 340.75 287 
Men 
Degrees 
Sum of of Mean _ 
Squares Freedom Square ! 
Between groups 11.43 5 2.29 
wll 
Within groups_ 151.11 143 1.06 
Total ___.-. 162.54 148 


*5% point = 2.27. 
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TABLE IV 


ANALYSIS OF VARIANCE: LIBERALISM SCORES 
ACCORDING TO MOTHER’S ATTENDANCE OR 
Non-ATTENDANCE AT COLLEGE 








Women 
Degrees 
Sum of of Mean 
Squares Freedom Square F 
Between groups 48 1 48 
Within groups— 329.51 273 1.21 
Total 329.99 274 
Men 
Degrees 
Sum of of Mean 
Squares Freedom Square F 
Between groups 9.75 1 9.75 
9.202 
Within groups— 143.43 135 1.06 
Total _.. 153.18 136 


21% point = 6.8. 


two categories. It should be noted that the 
second hypothesis tested was derived from 
the data of the sample, and is therefore open 
to question in spite of its statistical signifi- 
cance, unless it proves significant in a new 
sample.” 

The ease with which the initial classifying 
system may be telescoped in the use of 
analysis of variance represents an important 
advantage of this method for the empirical 
study of methods of classification. In this 
example it was found that for the particular 
sample, the essence of the grouping in terms 
of mother’s education was in the distinction 
between those who attended and those who 
did not attend college. By the use of quanti- 
tative criterion variables, it should be possible 
to determine the most economical system of 
classification for a variety of qualitative 
variables. 

The outcome of the combination of classifi- 
cations suggests a point which should be 
stated explicitly—the significance of an 
analysis of variance of the kind used here is 
not independent of the scheme of classifica- 
tion used for observations of the qualitative 
variable. If the @ priori hypothesis in the 
analysis of mother’s education in relation to 
liberalism had been the two-fold classifica- 
ton instead of the six-fold one, a significant 
relationship would have been found instead 
ol an indeterminate one. 

_, Cf. Peirce, Charles $., Collected Papers, vol. I, pages 39- 


or @ presentation of the dangers involved in forming 
potheses from the sample. 
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The analysis of variance of father’s edu- 
cation, when classified as in Table II, yielded 
mean square values which gave no evidence 
of a relationship between this variable and 
liberalism score. 

The analysis of variance of father’s occu- 
pation in relation to liberalism is presented 
in Table V. It is clear that among the men 
no well defined relationship exists. If, how- 
ever, for the women, the original code (which 
is modified from one used by the United 
States Employment Service) is telescoped into 


TABLE V 


ANALYSIS OF VARIANCE: LIBERALISM SCORE 
ACCORDING TO FATHER’S OCCUPATION 








Women 
Degrees 
Sum of of Mean 
Squares Freedom Square F 
Between groups 11.24 8 1.40 
1.19 
Within groups_ 329.51 279 1.18 
7... 340.75 287 
Men 
Degrees 
Sum of of Mean 
Squares Freedom Square F 
Between groups 5.25 8 66 
Within groups— 157.29 140 1.12 
Total ___._.. 162.54 148 


a threefold distribution as follows: (1) Pro- 
fessional, sub-professional, technical, ad- 
ministrative, supervisory, and unassigned; 
(2) Salespersons, clerical workers, service 
workers, and skilled craftsmen; and (3) Pro- 
duction workers and physical laborers, the re- 
sultant value of F is 4.42. For two degrees 
of freedom for the higher mean square and 
272 degrees of freedom for the smaller mean 
square, this is significant at the 5% level but 
not at the 1% level. 


SUMMARY 


This study presents several examples of 
the use of analysis of variance in dealing with 
the relationship between two variables, only 
one of which is quantitatively measured. 
With the systems of classification employed, 
none of the relationships turned out to be 
significant. Relationships between mother’s 


education and liberalism in men, and between 
father’s occupation and liberalism in women 
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are suggested. Evidence that the analysis of 
variance is not independent of the system of 
classification used is presented. 


VI. THE STUDY OF AN INTEREST 
QUESTIONNAIRE 
Reicn H. BittTNER AND EpWARD BorpIN 


The Progressive Education Association re- 
cently devised an interest questionnaire the 
function of which was to serve as one of sev- 
eral instruments for evaluating the programs 
of schools participating in experiments in 
progressive educational methods. The ques- 
tionnaire was designed to give some measure 
of the extent to which the curricula of the 
schools were meeting the interests of the stu- 
dents. Interests may be variously classified. 
In this instance, the attempt was made to 
build an instrument which would make ap- 
parent an individual’s pattern of interests in 
various areas. 

The questionnaire was developed through 
the cooperation of a group of secondary 
school teachers representing various subjects. 
This group gathered together a large number 
of statements of activities in which they had 
observed their students engaged, and which 
they judged to give evidence of the students’ 
interest in particular school subjects or other 
cultural areas. For instance, “To ride horse- 
back”’ was considered an evidence of interest 
in physical education or sports; “To take 
music lessons” was an indication of interest 
in music, etc. Many items were considered 
to evidence an interest in two or more areas. 
For example, “To discuss causes of war” in- 
dicated ‘Talking’, “Sociable”, ‘‘Mental’’, 
and “Social Studies” interests. The present 
form of the questionnaire embodies 300 
items. 

The questionnaire was designed to be scored 
to distinguish nineteen areas of interest. Items 
are grouped according to the areas in which 
the teachers judged them to be expressions of 
interest. As already noted, many items are 
considered to give an expression of interest 
in more than one area. The individual who 
marks the questionnaire responds to each 
item with “Like” or “Dislike”, or makes no 
response, which is interpreted in the scoring 
as “Indifferent”. Thus three scores—the 


number of “Likes”, “Dislikes” and “Indiffer- 
ents’’—may be found for each of the nineteen 
areas. The nineteen areas have been further 


| Vol. y \ 


grouped on a profile graph into fi, 
larger areas of interest named “Curricula; 
“Verbal”, “Aesthetic”, and “Sociable” for +h, 
facilitation of guidance, although no compox. 
ite scores are reported for these four area 

The present study is concerned with day, 
gathered during the use of this questionnajy, 
by the College of Education of The (bh; 
State University in the fall quarter of 19: 
The questionnaire was given to 462 freshm» 
for the purpose of evaluating their interes: 
as a basis for educational and vocation, 
guidance. (It is to be noted that this pr 
posed use of the questionnaire involves a ¢ji. 
ferent approach from that of the original jp. 
tention of the Progressive Education Associa. 
tion.) A number of questions arose as 
consequence of the administration of 
questionnaire. This study deals with three 
these questions. 


QUESTION I 


Are there any primary interest factors w 
derlying the nineteen areas of interest mea: 
ured by the questionnaire? 

The answer to this question has pract 
as well as theoretical significance. If it wer 
found that there was a small number 
fundamental interest factors, then it sh 
be possible to score the questionnaire 
these principal factors and to deriv: 
scores for any of the nineteen separate 
terests by a linear combination of the s 
for the principal factors. This would great) 
simplify the scoring of the questionnaire. |’ 
answer will also have a bearing on the us 
of the questionnaire in practical guidance 

A factor analysis of the correlation matri 
for “Likes” and “Dislikes” was made. 1h: 
“Simplified Multiple Factor Method” ce 
veloped by Thurstone' was used. For reasons 
which we will develop, the “Like” and “D's 
like” intercorrelations were analyzed sep* 
rately. 

Two factors are distinguished by “e 
analysis. Since the two factors reduced ' 
residual correlations to approximately w>' 
would have been expected by chance, " 
seems that it is not necessary to postulat 
more than two factors to account for the tao 
of intercorrelaions. The factor loadings © 
presented in Table II indicate that the 0™ 
factor is overwhelmingly important. le 
factor loadings for the first factor are all pos 


tor Method 


' Thurstone, L. L. A Simplified Multiple Fac! 
University of Chicago Bookstore, 1933, 25 pp 
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tive and relatively high for both “Likes” and 
“Dislikes”. The second factor loadings for 
both “Likes” and “Dislikes” with but few 
exceptions are relatively low and account for 
very little of the variance. 


At this point we may speculate as to the 
»robable nature of the two factors. The first 
factor loadings for “Likes” which seem most 
characteristic of the factor are derived in the 
order of their significance, from the interests 
labelled “Talking”, “Reading”, ““Humanitar- 
ian’, “Social Studies”, “English”, ‘Sociable’, 
“Mental”, “Writing”, and “Leadership”. It 
is difficult to assign a name to this factor but 
it seems probable that an interest in working 
with ideas may characterize this factor. 


(A+B 
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ture of the factors are speculative and merely 
tentative. These conclusions would require 
further study of an empirical nature for 
verification. 

The fact that many items are common to 
two or more interests raises another question. 
Are the factors found by means of the factor 
analysis only pseudo-factors resulting from 
the effects on the intercorrelations caused by 
certain items being common to the interests 
correlated? In order to get at this problem 
it was necessary to determine how much of 
the correlation was due to the presence of 
common items. 

A formula was developed for estimating 
the amount of the spurious correlation. The 
correlation between two tests is given by 
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The first factor loadings for “Dislikes” 
zive essentially the same picture as we found 
for “Likes”. “Theatre Arts” is added to the 
interests which appeared to constitute the 
first factor for “Likes”. We may again as- 
sume that an interest in working with ideas 
may characterize this factor. 

The second factor loadings for both 
Likes” and “Dislikes” indicate that it is not 
of great significance. However, we may con- 
jecture as to what this factor may be. The 
second factor loadings for “Likes” which are 
most characteristic of the factor are called 
Manipulative”, “Industrial Arts”, and “Fine 
\rts’. Table III shows that “Manipulative” 
s closely allied with “Industrial Arts” and 
Fine Arts’ as a result of the method of 
scoring items. We may postulate that the 
common factor here is probably an interest 
in working with things. 

The second factor loadings for “Dislikes” 
which are most definitive of the factor are 
called “Industrial Arts”, “Manipulative”, and 
_Science, Engineering and Mathematics”. 
Table IIT again shows that “Manipulative” 
s closely related to the other two by the 
method of scoring items. It seems that we 
may again assume that the second factor is 
characterized by an interest in working with 
things. 

It should be emphasized at this point that 
these considerations with regard to the na- 





-1) 


where a, b, . . . x are items of one test and 
A, B,. . . X are items of the other test. 

Now assuming that the o’s are all equal 
and the items are independent, equation (1) 
may be written 


Yaa + Tan + * + Txx 
VPVa 

where p is the number of items in the first 

test and g is the number of items in the 

second test. 

All numerator r’s are either 1.00 or 0.00. 
If the item is common to both tests, then 
r == 1.00 and if the item occurs in one test 
and not the other, then r = 0.00 in accord- 
ance with our assumption of independence of 
items. Then equation (2) may be written 
n 


VP Va 
in which n is the number of items common 
to the two tests. 

Table III shows the estimated spurious 
correlations determined when equation (3) is 
used. When these values are compared with 
the correlations shown in Table I, it is seen 
that there is some correspondence between 
the magnitudes of these correlations and the 
estimated amounts of spurious’ correlation. 
Now if there were a linear relationship be- 
tween these two series of values, we could 
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arch, 1940] 
-onclude that the individuality of the factors 
~ Jargely determined by the items which are 
-»mmon to the interests correlated. Figure 1 
“hows that some degree of linearity is evident 
«hen the intercorrelations of the interests 
cost characteristic of the first factor for 
“Likes” are plotted against their correspond- 
., estimates of spurious correlation due to 
‘mon items. However, it is also quite ap- 
sarent that the individuality of the factor 
< not solely determined by the spurious cor- 
elation, but that the factor has some sig- 
sincance in its own right. We can only 
-onclude that the factor is partially the re- 
it of the common items, and it is left for 
‘yrther study to determine the true nature 
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and magnitude of the factor. 
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Fig. 1. Intereorrelations of “Likes” Most 
Characteristic of the First Factor in Relation 
the Estimated Correlation Due to Items 
Common to the Variables Correlated 






















QueEsTION II 


\re the “Like” and “Dislike” scores posi- 
we and negative expressions of the same 
trait? 

The answer to this question will determine 
whether it is necessary to score the question- 
naire on both “Likes” and “Dislikes” or 
whether scoring on one will be sufficient. It 
was also intended that if we found only one 
score to be required some recommendation 
should be made as to which score was the 
more meaningful. It was these considerations 
that led us to analyze separately the “Like” 
and “Dislike” intercorrelations. 
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Table II shows that the factor composi- 
tions for “Likes” and “Dislikes” are similar 
to a marked degree. The first factor loadings 
for “Dislikes” are in general higher than 
those for “Likes” but there is apparently 
rather close correspondence between the two 
with respect to rank order. Rank order co- 
efficients of correlation were computed and it 
was found that p = .896 + .032 for the first 
factor loadings and .577 + .108 for the sec- 
ond factor loadings. Since “Like” and “Dis- 
like” scores are mutually exclusive responses 
to the same sets of items, it seems reasonable 
to assume that these results indicate that 
such scores are positive and negative expres- 
sions of the same trait. A really definite an- 
swer to this question can be obtained only by 
treating “Like” and “Dislike” scores in the 
same correlation matrix. 

For further evidence bearing on our ques- 
tion we turn to the correlations between 
“Likes” and “Dislikes” as shown in Table IT. 
These correlations (on the diagonal) are all 
negative, and rip = — .508. Now if these 
correlations were of the same order as the 
reliabilities of the variables we would have 
tangible evidence that “Likes” and “Dislikes” 
are positive and negative measures of the 
same trait. Estimated reliabilities as shown 
in Table II were computed using Kuder and 
Richardson’s “Case IV Method.’* The fact 
that the reliabilities greatly exceed the cor- 
relations between “Likes” and “Dislikes” 
might seem to indicate that scores for 
“Likes” and “Dislikes” are not positive and 
negative expressions of the same trait. How- 
ever, let us consider the correlations between 
“Indifferents” and “Likes” and ‘Dislikes”’. 
Table II shows that the correlations between 
“Likes” and “Indifferents’” are consistently 


higher than those between “Dislikes” 
and “Indifferents”’. In fact, ru1 = — .449, 
fpr = — .121, and rp == — .508. This in- 


dicates that “Like” and “Dislike” are at op- 
posite poles of a continuum but that the cor- 
relation between “Likes” and “Dislikes” is 
limited by the fact that “Indifference” also 
lies opposite to “Like”. Therefore we can 
not expect rip to be of the same order as the 
reliabilities, though in view of the data 
already cited, it is highly probable that 7,» 
corresponds closely enough to the reliabilities 


2? Kuder, G. F. and Richardson, M. W. “The Theory of the 
a of Test Reliability.” Psychometrika, 1937, 2, 
151- : 





a9 
276 


to add some further evidence that scores for 
“Likes” and “Dislikes” represent positive and 
negative expressions of the same trait. 

We may then conclude that the second 
question has been answered affirmatively. It 
also seems apparent from the factor loadings 
in Table II that “Dislikes” gives a better 
measure than “Likes”. This is especially true 
for the first factor. 


QuesTION III 


What is the significance of the “Indiffer- 
ence” score? 

There is some indication that the “Indif- 
ference” score yields a negative expression of 
the trait which is expressed positively by the 
“Like” score. By following the same reason- 
ing that we pursued in the analysis of the 
“Like’—‘Dislike” relation we find that the 
combination of an r.; = — .449, which is 
limited by the rip == — .508, and high re- 
liabilities for both “Likes” and “Indifferents”’ 
seems to give statistical evidence that “Likes” 
and “Indifferents” are to a certain extent 
positive and negative expressions of the same 
trait. 

The relation existing between “Dislike” 
and “Indifference” scores is manifestly differ- 
ent from that between “Like” and “Indif- 
ference” scores. As indicated by rp: == — .121 
there is very little relationship between these 
two series of scores. When rp; is compared 
with the estimated reliabilities as shown in 
Table II, there seems to be evidence to indi- 
cate that “Indifference” and “Dislike” 
scores are not measures of the same trait. 

This lack of relationship between “Indif- 
ference” and “Dislike” scores seems to point 
to the possibility that the “Indifference” 
scores may be a measure of a different 
psychological trait from that measured by the 
“Like” and “Dislike” scores. As we have 
shown, “Like” and “Dislike” scores are ap- 
parently positive and negative expressions of 
the same trait. Now it was also shown that 
statistical evidence points to the conclusion 
that “Like” and “Indifference” scores are 
positive and negative expressions of the same 
trait. However, if this were true we should 
expect that “Dislike” and “Indifference” 
scores would be closely related. Since that is 
not the case, it is highly probable that the 
relationship found between “Like” and “In- 
difference” scores is a function of the closed 
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system of scoring and not a true psychologic) 
relationship. It then appears to follow thy 
the “Indifference” scores may possibly repre 
sent a different psychological trait from thy 
measured by the “Like” and “Disliky 
scores. This supposition would necessari); 
have to be verified by further study. A fact;- 
analysis of the intercorrelations of the “|p. 
difference” scores on the nineteen posited jp. 
terests would give further evidence on thi 
point. 

It is to be noted that Table IT shows th 
estimated reliabilities for the “Indifference 
scores to be excessively high in comparisor 
with those of the “Like” and “Dislike 
scores. These excessively high estimated re. 
liabilities are probably due to the fact thy 
the standard deviations for “Indifference” x 
shown in Table II are in every case higher 
than those for “Likes” and “Dislikes”, This 
in turn results from the method of scoring 
which requires the sum of the “Likes”, “Djs. 
likes”, and “Indifferents” scores to equal 
constant, i. e. the total number of items. Ip 
such a closed system of scoring, the standar 
deviation of the “Indifference” scores equal 
the standard deviation of the composite 
scores obtained by adding the corresponding 
“Like” and “Dislike” scores. 


SUMMARY AND CONCLUSIONS 


The interest questionnaire recently de- 
veloped by the Progressive Education Asso- 
ciation was given to 462 freshmen in the (>- 
lege of Education of The Ohio State Univer- 
sity during the fall quarter of 1938. Three 
scores, namely the number of “Likes”, “Dis 
likes” and “Indifferents”, were obtained for 
each person on nineteen interests which the 
questionnaire was designed to distinguish 
The administration of the questionnaire gave 
impetus to the present study, in which ve 
have attacked the following questions: (1) 
Are there any primary interest factors under- 
lying the nineteen areas of interest measured 
by the questionnaire? (2) Are the “Like 
and “Dislike” scores positive and negative 
expressions of the same trait? (3) What § 
the significance of the “Indifference” score’ 

The statistical evidence shows that there 
are two primary interest factors which th 
questionnaire measures. The factor loadins 
for both “Likes” and “Dislikes” show that 
the first factor is of much greater importance 
than the second factor. An analysis of the 
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" «trinsic relation between interests resulting 
«om the fact that many items are scored on 
g rae or more interests shows that the first 
© factor is partially determined by the spurious 
correlation resulting from this method of 
ring. However, the factor seems to have 
some degree of individuality in its own right. 
Speculation as to the nature of the factors 
eads to the tentative conclusion that the first 
‘actor represents an interest in working with 
ideas and the second factor an interest in 
’ working with things. 
Statistical evidence points to the conclusion 
"that “Likes” and “Dislikes” are positive and 
_ negative expressions of the same trait. The 
lose correspondence between the factor load- 
nes for “Likes” and “Dislikes”, and a com- 
sarison of the negative correlations between 

Likes” and “Dislikes” with their estimated 
reliabilities, lead us to this conclusion. There 

"so seems to be adequate evidence to show 
that with the present method of scoring, the 

Dislikes” score gives a better measure than 
the “Likes” score. 

The study seems to indicate that the “In- 
ifference” score may represent a measure of 
, different psychological trait from that 
measured by the “Like” and “Dislike” 
cores. Our data indicate that “Like” and 
Indifference” scores may be positive and 
negative measures of the same trait. It also 
shows that “Dislike” and “Indifference” are 
not measures of the same trait. The negligible 
relationship between “Dislike” and “Indif- 
erence’, instead of the close relationship that 
would be expected on the basis of the 
previous finding that “Likes” and “Dislikes” 
are inverse measures of the same trait, indi- 
cates that “Indifference’’ may be a measure 
i a different psychological trait from that 
measured by “Likes” and “Dislikes”. The 
adication that “Likes” and “Indifierents” 
are positive and negative measures of the 
sume trait is apparently an outcome of the 
closed system of scoring. 

It is clearly evident that the questionnaire 
should not be considered to give measures of 
nineteen unitary and independent interests. 
‘ince the factor analysis shows that only two 
unitary interests are measured by the ques- 
tionnaire, it is clear that many of these nine- 
‘een interests are closely related and are in 
4 certain sense measures of the same thing. 
This fact should be taken into consideration 
using the questionnaire for guidance, and 
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it is suggested that a regrouping of the nine- 
teen interests on the profile graph to conform 
with the factor analysis might be in order. 


The results of the factor analysis also in- 
dicate that the scoring of the questionnaire 
could be simplified. It should be possible to 
score for the principal factors instead of the 
separate interests, and then derive the score 
for any given interest by a linear combination 
of the factor scores, using the proper weight- 
ings for the given interest. However, the 
practice of giving the items equal weights for 
several interests has been shown to influence 
not only the validity of the factors but also 
their composition. It is probable that with 
differential scoring weights for the items, a 
different and perhaps more valid factor pat- 
tern would result. It seems to be indicated 
that the question of differential weighting of 
items should be investigated before an at- 
tempt is made to develop a simplified method 
of scoring. 


The evidence that the questionnaire now 
measures preponderantly one unitary factor 
and to a lesser degree a second seems to in- 
dicate that careful consideration should be 
given to the selection and validation of items. 
It may be that in the population studied only 
two unitary interests are operating, but it 
seems more likely that there are others which 
the questionnaire does not measure. The 
uniqueness of some of the interests would 
also point in this direction. These other fac- 
tors, if found, would greatly extend the use- 
fulness of the instrument in guidance. 


It is apparent that “Like” and “Dislike” 
scores are not both necessary to the measure- 
ment of the interests with the present scoring 
system, but that one or the other will suffice. 
A consideration of the factor loadings for 
“Likes” and “Dislikes” indicates that the 
“Dislikes” scores should be used if a choice 
is to be made between them. 


Finally, it is strongly urged that a more 
intensive study be made of the “Indifference” 
scores to determine the nature of the 
psychological trait which they seem to meas- 
ure. In view of the fact that the closed 
system of scoring leads to complications in 
interpreting the “Indifference” variable, it 
seems advisable to investigate the possibilities 
of a different system of scoring which would 
eliminate these difficulties. 
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VII. MEASURING FRESHMAN 
COLLEGE STUDENTS’ ABILITY TO 
THINK IN TERMS OF SELECTED 
SOCIAL PROBLEMS 


WitiiAM J. JONES 


Included in the battery of diagnostic tests 
which all freshmen in the Ohio State Univer- 
sity College of Education took in conjunction 
with a first quarter Survey of Education 
course during 1938, was a test on social 
problems devised by the Evaluation Staff of 
the Progressive Education Association. It was 
the purpose of this test to secure evidence on 
various aspects of students’ thinking in situa- 
tions where social values and beliefs have a 
strong, if not determining, effect on the 
quality of thinking. 

The test was composed of six problematic 
situations dealing with such problems as the 
use of labor saving machines, housing, taxa- 
tion, racial discrimination, freedom of speech, 
and health security of workers in factories. 
Each problem was followed by three or four 
possible courses of action, one or more of 
which were to have been checked by the stu- 
dents as appropriate “solutions” of the prob- 
lem. Beneath the courses of action about 20 
or 25 typical reasons were supplied, some of 
which might be used to justify the course (or 
courses) of action checked by the students. 
For each of the six problems in the test, the 
students were to check appropriate courses 
of action and to support their conclusions 
with reasons by marking those which they 
felt to be correct. 


Typical of the kinds of problem situations 
included in the test is the one which follows: 


PROBLEM I 


Cotton has been picked by hand, which 
is a slow and expensive process. Recently, 
the Rust brothers invented a machine to do 
this work. It would pick in 74 hours as 
much cotton as one handpicker could pick 
over a whole season of eleven weeks. The 
cost of production of cotton could be reduced 
from $14.52 to $3.00 per bale. Today this 
machine has not been placed on the market. 


Directions: Check the course (or courses) of 
action which you think should be followed 
in this situation. 
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Courses of Action: 


A. 





——B. 


The invention should be made ay). 
able for unrestricted manufacture ay 
sale of the machine. 
The machine should be manufactyre: 
and sold under some form of go. 
ernment control and provisions ma¢: 
for establishing in other jobs the c: 
ton pickers who are thrown oy 
work. 


——C. Workers and cotton growers should 


—D. 


form a cooperative and should ys 
the profits from it to take care 
the people thrown out of work }y 
the machine. 

The machine should not be put 
use at the present time at al! 


Directions: Choose the reasons which you 
would use to support your course (or 
courses) of action and check them jn th 
space provided. 


Reasons: 





== 3O. 


I. 


. Uncontrolled use 


In business the efficiency of produc- 
tion should be considered ahead of 
anything else. 

of the machine 
would give an advantage to a fen 
large landowners in the South over 
the poor landowners. 


. The labor problems of the South ar 


for the South to solve. 


. One should consider human effect 


ahead of business efficiency. 


. Uncontrolled introduction of a labor 


saving machine throws large num- 
bers out of work. 


. Men who are clever enough to in- 


vent useful machines should have 
the opportunity to make profits on 
those inventions. 


. When labor saving machines are i- 


troduced, the people who are dis 
placed by them find work in making 
the new machines. 


. People who are directly involved ® 


cotton production can better plat 
for the use of the machine. 


. Machines have worked too much 


harm on people already. 
People thrown out of work by the 
introduction of the mechanical “© 
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ton picker will be on relief rolls and 
thus add to the financial burden of 
the government. 
x * * 
_ Society should not be deprived of 
anything that might improve its 
work and the products it uses. 


Some of the statements above represent 

ints of view which expressed a concern for 

mocratic and human values, freedom of 
-neech and social control, while others de- 
‘ended special privilege and individualism as 
values. The tests were not scored on a right 
od wrong basis, but the reporting of student 
serformance was in terms of the comprehen- 
syeness of the students’ thinking (their abil- 
ty to support courses of action with sound 
~asons), as well as the validity, relevance, 
ind consistency of the reasons chosen in 
verms of the courses of action which were 
checked. In addition to this, the dominant 
nattern of values expressed through the stu- 
dents’ choice of conclusions and reasons was 
escribed. 

lable | contains the median scores and the 
range of the scores for the entire group of 
545 education freshmen who took the test. 

This table indicates that, on the average, 
the students chose six courses of action (one 
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for each problem), and that they checked 42 
reasons to support their views, of which 31 
were sound reasons. The comprehensiveness 
ratio of sound reasons per conclusion was 
4.5. That the students’ thinking in the field 
of social values is somewhat confused is evi- 
denced by the fact that of the 42 reasons 
which the typical student checked, six of 
them on the average actually contradicted 
the courses of action which the student 
checked. The mistake is akin to supporting 
black with white. The students also were 
guilty of using irrelevant and invalid reasons 
to support their points of view. Table I fur- 
ther indicates that the students’ patterns of 
value are highly democratic, with a slight 
tendency toward compromise, but by and 
large, the students chose no undemocratic 
courses of action, and only four out of a 
possible total of 36 undemocratic reasons, 
i. e., special privilege and individualism. 

In addition to presenting this picture of 
the democratic pattern of values rather com- 
prehensively supported by sound reasons, 
which characterized the social attitudes of 
the group of over 500 freshmen, this report 
has the additional purpose of presenting the 
inter-correlations between the students’ 
performance on the sixteen parts of the test, 
as well as the relationships between the sev- 


TABLE I 
MEDIANS AND RANGES OF SCORES ON A SOCIAL PROBLEMS TEST 
Highest 
Median Range Possible 
Score 
Comprehensiveness 
1. Number of courses of action checked__- 6 3-14 22 
2. Number of reasons checked_____________ 42 13-125 144 
3. Number of sound reasons checked________- : 31 6-85 144 
4. Average number of sound reasons per conclusion - 4.5 0. 5-12. 7 fie 
Confusion of Thinking 
5. Number of reasons inconsistent with course of action : 6 0-42 144 
6. Per cent of irrelevant reasons___________- 22 0-67 100(9) 
7. Per cent of invalid reasons__...._______________- 10 0-80 100(10) 
DOMINANT VALUES & VIEWPOINTS 
Values Expressed Through Conclv :ions 
8. Number of democratic conclusivns___--_-_-_-_- ; 4 0-9 9 
9. Number of undemocratic conclusions___-_---___--- 0 0-4 7 
10. Number of compromise conclusions. ___________- 2 0-5 6 
Values Expressed Through Reasons 
ll, Defense of special privilege. __............_--- 3 0-25 27 
12. Defense of human values___._.......-------.------ 14 2-28 28 
13. Defense of social control in behalf of human values and 
0 AT TTS 3 0-7 7 
14. Defense of individualism_____.___._.......--.------ 1 0-8 9 
15. Defense of democratic principles__.____________..-_-- 10 1-24 24 
16. Defense of compromise for sake of expediency ____ _- - _- 1 0-9 9 
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RELATIONSHIP BETWEEN SCORES ON A SOCIAL PROBLEMS TEST AND OTHER FACTORS 


Autumn Grades.____ 
Survey Grade____-_ 


Rank in Class 
Intelligence 
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eral parts of this test and such data as rank 
in high school class, intelligence, and 
academic achievement as measured by 
grades. 

An analysis of such a table of intercorrela- 
tions would provide one basis for answering 
such questions as the following: 

Who is the more comprehensive in his 
thinking, the person with the democratic or 
the undemocratic outlook? 

Is the tendency to make one type of error 
related to the tendency to make other types 
of errors? 

Is one who holds a democratic point of 
view more likely sometimes to take a com- 
promise position than one who holds an un- 
democratic point of view? 

Which outlook do the students who make 
the highest grades in school and the highest 
scores on an intelligence test take? 

Table II presents the inter-correlations be- 
tween these variables. It indicates that those 
students who chose democratic courses of 
action tended less often to choose compromise 
positions than did those students who chose 
undemocratic courses of action. The table 
further shows that the tendency to make one 
type of error possible in the test is associated 
with the tendency to commit other types of 
errors possible in the test. The tendencies 
to use inconsistent and invalid reasons seem 
to be the more closely associated. To facil- 
itate interpretation this master table has been 
subdivided into significant parts. 

Table III presents the relationships which 
exist between the sixteen test variables and 
the following: (1) the student’s rank in his 
high school class as measured by a lowest 
third, middle third, and upper third classifica- 
tion; (2) the student’s intelligence as meas- 
ured by the Ohio State Psychological Exam- 
nation; (3) academic achievement as meas- 
ured by his first quarter grade-point average; 
and (4) grades in the Survey of Education 
course when recorded on a letter basis from 
A to E. 
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The interesting fact about Table III is thy 
there are no high correlations, either negatiy. 
or positive, included in it. The r of .33 ty. 
tween intelligence and the comprehensivenes 
score is the highest coefficient of correlation, 
found in Table III. 

Although the coefficients of correlation ar: 
small, it is nevertheless significant to pot 
the pattern of positive and negative correl). 
tion which exists between the columns. With 
the exception of columns 1 and 2, the signs 
of the quantities within a given colur 
all the same. It is significant to note that the 
students who chose democratic courses 
action (column 8), and supported them cop- 
sistently with reasons which defended humar 
values (column 12), social control (colum 
13), and democratic principles (column 1; 
tended to be those students with higher jp. 
telligence test scores (row 2), higher stand. 
ing in their high school graduating classes 
(row 1), and who secured the better grade 
(rows 3 and 4). This conclusion is supported 
not so much by the fact that the correlation 
coefficients are high, as by the fact that for 
each column mentioned above the r’s are in- 
variably positive across all of the four rows 
The fact that there is a consistent invers 
relationship between the tendency to chec! 
undemocratic (column 9g), and compromise 
(column 10) courses of action, utilizing rea- 
sons defending special privilege (column 11 
individualism (column 14), and compromise 
(column 16), and the tendency to rank high 
in one’s high school graduating class, having 
a high intelligence test score, and the ability 
to secure high grades, tends to emphasize th 
relationships noted above. 

Table IV shows the relationships between 
the various courses of action and the difierent 
types of reasons which were used to suppor! 
the courses of action. The table indicates 
that there is a fairly high correlation (rang- 
ing from .38 to .43) between the tendency 
to choose democratic courses of action an 
to support them with reasons which affirm 
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TABLE IV 
RELATIONSHIP BETWEEN VARIOUS COURSES OF ACTION AND CERTAIN KINDS OF REASONS 


. Special Human 

Courses of Action Privilege Values 
Democratic ............._ —.14 43 
Undemocratic ....._______ ot —.18 
Cemmpeennie® .....cncccns 36 —.08 


Reasons 
Social Individ- Democratic Com 
Control ualism Principles — promt 
40 05 38 —02 
—.12 .27 .02 05 
—.20 12 —.11 25 


oo — -e ees ee 
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emocratic principles and defend human 
“jues and social control. More significant 
nerhaps are the positive correlations between 
‘he tendency to choose undemocratic courses 
‘ action and the tendency to choose reasons 
yhich defend individualism and _ special 
orivilege as social values. Interesting, too, 
= the tendency for those who chose reasons 
farming special privilege to choose com- 
oromise courses Of action. 

lable V indicates the relationships between 
the tendency to make certain types of errors 
and the tendency to choose certain courses 
of action and various types of reasons. The 
table shows that there is almost no relation- 
ship between the kinds of courses of action 
one chooses and the kinds of errors in think- 
ing which are made. The table further in- 
dicates that those students who chose reasons 
defending special privilege, individualism, 
and compromise positions tended to be 
more inconsistent in their thinking than 
those who chose reasons which supported 
democratic principles, human values, and so- 
cial control. Table V likewise shows that 
there is a fairly high correlation (.52) be- 
tween the use of “human values” reasons and 
the tendency to employ reasons which are ir- 
relevant to the courses of action selected. 
Row 3 in Table V shows that those students 
who chose reasons defending special privilege, 
individualism, and democratic _ principles 
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tended more often to use invalid reasons than 
those students who chose other types of 
reasons. 

Table VI shows the inter-relationships be- 
tween the use of the various types of social 
reasons included in the test. It indicates that 
the tendency to employ reasons which sup- 
port special privilege is associated with the 
tendency to employ reasons which support in- 
dividualism as a basis for social action. Table 
VI also shows that those students who chose 
reasons which defended democratic principles 
likewise tended to choose reasons which 
affirmed social control as valid social values. 

This study has indicated that the College 
of Education Freshmen at the Ohio State 
University tend to be democratic in their so- 
lutions of social problems, and that they are 
able, on the average, to supply four or five 
sound reasons for each of their courses of 
action. The pattern of values exhibited in 
the types of reasons which they employ ex- 
pressed a concern for the defense of funda- 
mental democratic principles, human values, 
and social control. The _ inter-correlations 
show that the “better” students (high grades, 
high intelligence test scores, and high rank in 
high school graduating class) tended to 
choose democratic courses of action and 
democratic reasons more often than they 
chose undemocratic or compromise courses of 
action and reasons. 


TABLE V 


RELATIONSHIP BETWEEN VARIOUS KINDS OF ERRORS AND DIFFERENT TYPES OF COURSES 
OF ACTION AND SOCIAL REASONS 


Reasons 
Errors Courses of Action Demo- 
Demo- Undemo- Com- Special Human _ Social Individ- cratic Com- 
cratic cratic promise Privilege Values Control ualism Principles promise 
Inconsistent .03 13 13 50 .29 22 44 36 46 
Irrelevant__ .17 06 —.06 05 52 AT 17 30 .20 
Invalid 10 17 .02 35) .26 oe 31 .30 .22 
TABLE VI 
INTER-RELATIONSHIPS BETWEEN THE USE OF VARIOUS TYPES OF SOCIAL REASONS 
Reasons 
Reasons Special Human Social Individ- Democratic Com- 
Privilege Values Control ualism Principles promise 
Special Privilege __.____ 
Human Values _________ 01 
Social Control _...______ 16 41 
Individualism __________ 49 .06 10 
Democratic Principles ___ __.30 40 58 .28 
“mpromise ............ 44 .20 10 29 12 
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VII. RELATIONSHIPS OF TEST 
SCORES OF EDUCATION 
COLLEGE FRESHMEN 
TO GRADES IN 
SELECTED 
COURSES 
ARTHUR C, CAHOW 


The purpose of this study is to determine 
the relationships which exist between the 
marks in various courses and the scores ob- 
tained from the various tests. 

Course marks were available for English 
401, Composition; Botany 4o1, First Course 
in Botany; Psychology 401, Elementary Psy- 
chology; History 403, American History to 
1852; Geography 4o1, Elementary Geog- 
raphy; Music 423, First Course in Pub- 
lic School Music; Fine Arts, Freehand Draw- 
ing, and Zoology 401, Elementary Zoology. 
Inasmuch as this investigation presupposes 
some degree of choice of the courses selected, 
English 401 was eliminated because it is re- 
quired of all students; Zoology 401, Psychol- 
ogy 401, Geography 401, Music 423, and 
Zoology 401 because the number of cases was 
small, being considerably less than fifty in 
each instance. 

The courses selected for this study were: 
A) Botany 4o1, First Course in Botany; B) 
History 403, American History to 1852; and 
() Fine Arts 421, Freehand Drawing. His- 
tory 403 might be defined as a partial elec- 
tive, since it is required of students in the 
Elementary Teachers’ curriculum of the Col- 
lege of Education. These three courses are 
assumedly quite different. One is a natural 
science, one a social science, and the other a 
skills course in an esthetic field. With such a 
selection of courses, the liability and asset 
values of the variety of backgrounds reflected 
by the tests may be investigated. 

The test scores used more selected from 
those described in the preceding studies. Not 
all of the subscores were used. Selection was 
made on the basis of the statistical evidence 
already presented and on the judgment of 
those using the test results. 

Scores of the O. S. U. Psychological Test 
are expressed in percentile ranks, scores of 
other tests in percentage, and course marks 
as letters A, B, C, D, and E. Hollerith 
technique was employed for tabulation. 

Correlation coefficients were computed for 
each of the test scores and grades for each 
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of the selected courses. The means ang 
standard deviations were obtained for the tes; 
scores of those who did and of those who dj 
not elect each course. 

The essential results of this study ar 
shown in two tables. Table I shows correl;. 
tions of each of the three selected course 
with each of the sub-tests used in this sty¢; 
Table II shows the means and standard deyi:. 
tions of the scores on each test for those wh 
elected each of the three courses and for thos 
who did not elect these courses. 

In general, correlation coefficients are 
low numerical value. However, assuming thy 
validity of these tests and the represents. 
tiveness of the samples, those to be discussed 
here are significant with respect to a group 

One might arbitrarily select as a significan: 
value of a correlation coefficient only on 
which has less than one chance in one hur- 
dred of being drawn from a universe where 
the true value of the correlation coefficient is 
zero. Any correlation coefficient which is a 
least 2.236 times the standard error of a cor- 
relation coefficient of zero, with the sam 
number of observations, may then be consid- 
ered significant. The minimum significant 
value for the Botany correlations is 0.17¢ 
for the History correlations, 0.284, and for 
the Fine Arts correlations, 0.293. 

For convenience in presenting the results 
the conclusions will be discussed in terms 
the variables composing the study. 


O. S. U. PsycHoLocicAL EXAMINATION 


The coefficient of correlation of the Psy- 
chological Examination with Botany graces 
is 0.526 and with History grades, 0.5%! 
These coefficients are among the highest 
found in this study. One might be inclined 
to believe, a@ priori, that the numerical rela- 
tionship should be reversed, inasmuch as the 
Botany grades are computed from the scores 
of five standardized tests, whereas the His 
tory grades are computed from the results on 
subjective type tests and grades on term 
papers. The correlation of the Psychologica 
Test and Fine Arts grades is 0.270. 4 
popular belief is that students who ele 
“manual” or “skills” courses are of lower !0- 
telligence than are those who major © 
academic work. Intelligence is here defined # 
that trait which is measured by the 0. S. ¢. 
Psychological Test. Examination of the meas 
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CORRELATION COEFFICIENTS OF TEST SCORES AND COURSE GRADES 
History 403 


0. S. U. Psycholozical Examination 


Social Problems Test 
No. of Sound Reasons__-_ ~~ -- 
Average No. of Sound Reasons 
No. of Inconsistent Reasons_. 
No. of Democratic Conclusions _ - 
No. of Compromise Conclusions 
Defense of Special Privilege 
Defense of Individualism 
Defense of Democratic Principles 
Defense of Compromise 


Social Attitudes Test 
Liberalism 
Democracy 
Nationalism - 
Militarism 
Total 


Conservatism 
Democracy 
Nationalism 
Militarism 
Total 


Uncertainty 
Democracy 
Nationalism 
Militarism 
Total 

Consistency 
De mocracy 
Nationalism 
Militarism 
Total 


Interpretation of Data Test 
General Accuracy 
Relative Values (Total Score 
Insufficient Evidence 
Obviously True or False 
Caution—Understatement 
Beyond the Facts 


Contemporary Affairs Test 
Total Score 


Part I, Political—Social 
International Events 
Total Score—Part I 


Part II, Contemporary Culture 
re 4 Pp he 
Music and Radio._.._______- 


Movies 





TABLE I 


Botany 401 


. 526 


273 
. 230 
120 


.110 


. 026 
. 090 
. 061 


. 176 


016 


247 
.318 
230 
293 


. 132 
. 268 
.173 


.201 


. 140 
088 
. 128 
.144 


. 367 


. 304 
. 230 
. 300 


407 
. 170 
. 438 
-. 264 
. 154 


. 108 


. 190 


. 230 


. 106 
. 073 


. 060 
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. 580 


223 
424 
. 362 
. 050 
062 
. 157 
. 224 


. 024 


066 


049 
109 


327 
. 198 
. 284 
. 352 


. 584 


. 522 


. 371 


. 093 
. 442 


. 306 


. 227 
. 330 


.170 


.130 


. 202 


Fine Arts 421 


. 270 


185 
.148 
. 142 
. 209 
.023 


136 


.198 
. 072 


104 


. 286 


050 
199 


. 194 
. 210 
. 194 
-. 147 
—.217 


. 104 
. 206 


. 156 
. 049 
. 074 
. 143 
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TABLE I 


CORRELATION COEFFICIENTS OF TEST SCORES AND COURSE GRADES—Continued 


Interest Questionnaire 


Science, Engineering, 


and Mathematics 


Social Studies 
English _ - 
Fine Arts 
Leadership 
Humanitarian 
Sociable_- 
Withdrawal 
Reading 
Talking 


TABLE II 


Botany 401 


. 051 
.191 
. 023 
.119 
. 193 
. 145 
. 054 
. 094 
. 144 
. 176 
. 050 
. 094 
. 128 
. 183 
. 086 
. 189 
.114 
-.214 

. 238 

. 125 


[ Vol. 5, No . 


History 403 Fine Arts 42) 
. 184 093 
.028 230 
. 030 —, 158 
. 049 —. 068 
. 038 —,_ 030 

—.016 —., 237 
—, 206 . 301 
. 070 —, 679 
.O11 —. 261 
—.101 —. 145 
. 032 —. 106 
. 0388 ——, 098 
—. 068 —, 295 
. 094 —. 148 
—.017 —.015 
.018 —. 078 
—. 125 . 026 
—., 044 .170 
—. 026 —, 252 
—. 062 —, 042 


MEANS AND STANDARD DEVIATIONS OF THE TEST SCORES OF THOSE WHO Dip ANpD 
TuHose WuHo Dip Not ELEctT CERTAIN COURSES 


Did Not Take Course 


Verbal Intelligence _. 


Social Problems 


No. Sound Reasons 


Av. No. Sound 
Reasons 

No. Inconsist. 
Reasons. ___- 

No. Democratic 
Conclusions 

No. Compromise 
Conclusions 

Def. Spec. ~ 
Privilege. __ 

Def. Individualism 

Def. Dem. 
Principles. ____- 

Def. Compromise 


Social Attitudes 


Liberalism 
Democracy ___- - 
Nationalism ____ 
Militarism _ _ 
wan omen : 


Conservatism 
Democracy ._. _- 
Nationalism _ __- 
Militarism _ _ _ _- 
Weiiwexkesas 


Botany 
401 

M o 
60.8 25.6 
29.9 8.3 
1.0 3 
18.1 14.7 
4.3 1.4 
2.0 1.0 
3.5 3.4 
0.9 1.0 
9.6 3.8 
1.8 1.8 
65.0 10.3 
60.1 15.2 
61.0 13.6 
60.0 10.6 
22.0 8.4 
23.8 12.3 
25.6 10.5 
22.3 8.4 


Took Course 


History Fine Arts 
403 421 
o M o 

53.8 29.0 65.1 22.9 
29.9 8.8 28.8 8.7 
3.8 1.2 4.0 1.3 
18.3 33.2 27.1 48.7 
4.4 1.4 4.1 1.5 
6 £0 9 2:2 
3.8 3.5 4.0 3.3 
10 1.3 0.8 0.9 
10.1 4.4 9.7 4.2 
19 2.0 2.0 1.8 
63.3 9.6 63.2 9.3 
58.8 16.0 59.4 14.3 
59.3 15.9 63.2 12.3 
58.3 10.5 58.0 9.5 
23.0 9.4 23.0 7.5 
24.0 11.3 24.8 12.3 
26.0 12.5 24.4 10.6 
22.9 8.3 23.8 7.8 


Botany History 
401 403 

M o M o 
61.0 26.3 62.3 25.3 
30.4 10.4 30.2 9.3 
4.0 1.3 4.0 1.2 
19.3 14.7 18.9 15.1 
4.5 1.5 4.4 1.5 
1.9 a3 2.0 1.1 
4.8 4.0 4.4 3.9 
1.0 1.9 i ae oe 
10.3 4.4 10.0 4.8 
fe 80 E&P? &8 
64.5 9.6 64.9 9.9 
60.2 16.3 60.4 15.9 
63.2 7.0 62.2 8.1 
60.0 10.9 60.2 10.8 
20.2 9.1 20.5 8.7 
20.5 14.5 21.2 12.3 
24.6 10.8 24.8 10.3 
21.1 10.0 21.3 9.7 


Fine Arts 
421 
M z 
60.3 26.4 
30.4 93 
4.0 1.2 
20.2 14.9 
4.4 1.4 
Se EI 
4.4 3.9 
1.0 1.2 
10.1 4.2 
1.9 19 
64.9 9.9 
60.3 16.2 
61.5 9.3 
60.2 10.9 
20.5 9.0 
91.1 12.1 
25.0 10.6 
21.4 9! 





/ 





Botany 
401 
M o 
Uncertainty 
Democracy -- - -- u.T 63 
Nationalism_... 20.2 12 
Militarism - - - - - 16.4 10 
, eee 19.8 10. 
Consistency 
Democracy- -- -- 59.6 12. 
Nationalism.... 55.3 18. 
Militarism - - - - - 59.9 15 
Totel....-..<.. G& ¥0. 


interpretation of Data 


General Accuracy 


Relative Values. 53.0 
Insufficient 

® Evidence. ____ 50.3 

Obviously T or F 59.5 

Understatement 29.4 
Beyond the 

Facts........ 43.2 

‘ontemporary Affairs 
Total Score__._.-. 70.2 


Part I, Pol.—Soce. 
Internat. Events 11.9 
0 


Total Score____- 28 
Part II, Cont. Culture 
eee | 
Musicand Radio 9.0 
Movies. __..... 26.7 
Total Score__-__- 42.0 
nterest 
Science, Eng’r., 
Math. _L 16.3 
D 10.6 
Social Studies._L 19.0 
D 5.7 
English... ___- L 20.2 
D 6.7 
Fine Arts......L 19.5 
D_ 3.9 
Leadership____L 8.6 
‘ D_ 3.0 
Humanitarian- 
mm_........b 82.9 
D 2.4 
Sociable... __ L 30.6 
D = 2.9 
Theoretical. __L 9.4 
D 4.3 
Reading..___.L 34.0 
D 9.9 
Talking -_L 18.7 
D_ 6.0 
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1 
1 
1 
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TABLE II 


MEANS AND STANDARD DEVIATIONS OF THE TEST ScoRES OF THOSE WHO DiID AND 
THOSE WHO Dip Not ELEcT CERTAIN CouRrsES—Continued 


Took Course 
History 
403 
M o 


15.7 
21. 
18. 
20. 


“Ne 
—s 

son 

onow 


o 

adh call at 
Coon 
~ 

tb 
Como 


14. 
19. 
11. 
15. 


50. 
45. 
28. 
45. 


CHS ww 
> 
ao oonw 


bo 


70.0 28.7 


12. 
27. 


ao_- 
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on 
oO 


14. 


26. 
41. 
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no NoOere 
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Fine Arts 
421 


_ 
er! 
Oro 


on 
ed sell sa 
10S 


45. 
29. 
46. 


o 
ad : 
“a Of © 


69.8 


— 
_— 
ao 
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oS 
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PASANS=3 
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15. 
14. 
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_ 
"P00 29.9 GET LOE NOG NDH E1~1 91 G0 G0 ww 


more 


Co AFD Ow 


S390 60m 
Wona 


aon 


AK OSCE NOHS CIHOCWOOISCORD 


Botany 
401 
M o 
18.5 8. 
23.2 16 
18.0 11 
21.4 11 
59.0 12 
47.4 22 
41.2 28 
58.5 11 
52.3 12. 
46.9 17. 
59.4 14. 
26.5 9 
45.0 13. 
72.0 30. 
12.8 6. 
30.7 17. 
14.3 5. 
8.7 3. 
24.3 10 
40.6 18 
19.6 10. 
8.9 7. 
19.4 8. 
5.4 5. 
19.3 8. 
6.2 5. 
19.1 ‘6. 
4.1 3. 
8.5 4. 
3.7 3. 
12.8 5. 
3.3 &. 
23.7 7. 
3.3 3. 
10.7 5. 
3.8 3. 
34.0 12. 
9.5 9. 
19.0 7. 
6.0 5. 
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Did Not Take Course 
Fine Arts 
421 


DOAAD 


cor) 
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History 
403 
M o 
17.8 8. 
21.6 13. 
17.4 11. 
20.9 11 
59.1 12. 
49.3 22. 
45.9 27 
58.8 11 
63.3 11. 
48.6 16. 
69.8 14. 
41.8 9. 
42.2 12. 
71.6 30. 
12.6 6. 
31.2 17. 
14.3 ‘5. 
8.9 3. 
25.7 9. 
41.0 18. 
18.8 10. 
oe? 
19.0 8. 
6.4 5. 
19.4 8. 
6.0 65. 
19.2 6. 
3.9 3. 
8.4 4. 
3.2 8. 
12.6 4. 
2.3 3. 
29.8 7. 
3.2 3. 
10.5 5. 
3.9 3. 
34.1 12. 
9.4 8. 
19.0 6. 
5.9 5. 
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Psychological Test scores of Fine Arts stu- 
dents, in Table II, reveals that this assump- 
tion is not justified. 


SocrAL PROBLEMS TEST 


The correlation between Botany grades and 
scores on Total Number of Sound Reasons is 
0.273, and with Average Number of Sound 
Reasons per Conclusion is 0.230. The cor- 
relation between History grades and scores on 
Average Number of Sound Reasons per Con- 
clusion is 0.424, and with Number of Incon- 
sistent Reasons is — 0.362. While scores of 
reasoning skills correlate to some extent with 
History grades, correlations with scores of 
“point of view variables” such as Democratic 
Conclusions, or Compromise Conclusions are 
good approximations to zero. It might seem 
that in any consideration of social problems, 
point of view should be a concomitant of 
reasoning. 


SocraL Attitupes TEST 


Significant positive relationships exist be- 
tween grades in both Botany and History and 
scores indicating liberalism. Possibly the bet- 
ter students score higher on liberalism because 
they are more capable thinkers; perhaps they 
are more easily indoctrinated. 


Of interest is the correlation coefficient, 
0.352, of Fine Arts grades and scores on lib- 
eral attitude toward nationalism. This may 
be a chance relationship such as was obtained 
between the hardness of asphalt pavement, 
over a period of years, and the salaries of 
school superintendents. Perhaps there is a 
more plausible explanation. Much of the 
recognized art originated in Europe. The 
fact that it is recognized, accepted, and in 
many cases emulated, would seem to indicate 
that the better art students are not averse 
to accepting foreign ideas, particularly if they 
pertain to Art. Since statements in the na- 
tionalism sub-test are based upon the ac- 
ceptance or rejection of “foreign” rather than 
“idea”, it does not seem illogical to assume 
that the better beginning art students will 
tend to interpret these statements in terms 
of art. 

The scores on consistency of attitudes 
show significant relationships with grades in 
all three courses. 
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INTERPRETATION OF DATA Test 


The correlation of the score on Relative 
Values with History grades is 0.584, and with 
Botany grades, 0.407. These relationships 
are noticeable both because of their size rel). 
tive to the correlations of other test score 
with course grades, and because the Botany 
Department stresses interpretation of data sx: 
a teaching objective. 

History grades correlate more highly with 
scores on ability to recognize insufficient ev. 
dence; likewise with the scores on ability ; 
recognize probably true or probably fale 
items. Botany grades correlate more high) 
with the scores on ability to recognize tr: 
and false statements. 


CONTEMPORARY AFFAIRS TEST 


The highest correlations of the Contemy 
rary Affairs variables are with History 
grades. The total scores and the Part | 
Political-Social score, have correlations 
0.306 and 0.330, respectively. Chronolog, 
excepted, the content of History 40; 
similar to the content of the Contemporary 
Affairs Test. Thus it is not unreasonable : 


do well in the other. 

For Botany, the Political-Social score and 
the score on International Events have sig- 
nificant correlations. It may be observed that 
both of these correlations are higher than for 
Total Score. 

Noticeable is the absence of significant re- 
lationships between course grades and scores 
on Contemporary Culture, although there is 4 
positive correlation between History graces 
and scores on this sub-test. 


INTEREST QUESTIONNAIRI 

Botany grades show six significant correl- 
tions with Interest Questionnaire scores. Dis 
like of Science, Engineering, and Mathema'- 
ics, —o.91; Like of English, 0.193; D's 
like of Reading, — 0.214; Dislike o! © 
ciable Interests, — 0.183; Dislike of \Vith 
drawal Interests, — 0.183, and Like of Tall- 
ing and Conversation, 0.238. In such a ( 
lection of correlations, a tolerance ol, ratic’ 
than an interest in Science, Engineering, 2° 
Mathematics might be implied. 

The absence of a high correlation betwee? 
History grades and scores showing an inter 
est in the Social Studies, together with the 
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lack of positive relationship of Botany grades 

to scores on Interest in Science, Engineering, 

and Mathematics, might raise a question as 
+) whether, under present conditions, interest 

in success is not more pertinent to good 
- orades in an academic subject than is inter- 

est in the subject, per se. Further, do teach- 

ers consider interest to be a device for raising 
the achievement level of a group, rather than 
, concomitant of good teaching? It can be 
concluded that interest, as measured by the 
Progressive Education Association question- 
naire, is not absolutely essential to success in 
botany gor and History 403. 

The correlation of Fine Arts grades and 
scores on Interest in Fine Arts is 0.301. There 
‘s no evidence to indicate whether interest is 
, factor in success or success is a factor in 
nterest. 

It may be noted that most of the “Like” 
correlations, involving social interests, are 
positive, and correspondingly the “Dislike” 
orrelations are negative. These relationships 
illustrate the fallacy of a popular conception 
that a student’s “sociability” is in inverse 
proportion to his scholarship. From the 
standpoint of counseling, the reverse is 
worthy of consideration. 

lhe most significant relationships indicated 
oy this study are between Botany and the 
0. S. U. Psychological Test, r 0.526; His- 
tory and the O. S. U. Psychological Test, 
r= 0.581; Botany and the Interpretation of 
Data Test, r==0.407, and History and the 
Interpretation of Data Test, r 0.584. No 
attempt has been made to select a battery 
of tests most predictive of marks in each 
course. Statistically, the best tests were the 
Q. 5. U. Psychological Test and the Inter- 
pretation of Data Test. 


IX. A STUDY OF CHARACTERISTICS 
OF EDUCATION FRESHMEN 
WHO ENTERED OHIO 
STATE UNIVERSITY 
IN 1938 


Wape S. AMstTuTz 


. lt is proposed in this study to investigate 
‘he relationships between various test scores 
and the grades received in the freshman 
courses. It is recognized that college marks 
are not the sole desirable evidence of satis- 
‘actory progress toward adequate profes- 
‘ional and personal adjustment. Such marks 
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constitute only one of the possible criteria. It 
is hoped that this study, limited to a specific 
problem, will be the starting point for further 
studies involving other criteria, not now 
available. 

The five tests listed in the introduction to 
this series of articles, namely the Cooperative 
Contemporary Affairs Test, an Interpretation 
of Data Test, an Interest Questionnaire, a 
Social Problems Test, and a Social Attitudes 
Scale, as well as the Ohio State University 
Psychological Examination constitute the 
battery of tests which are the basis for this 
study. 

The point-hour ratio for the autumn quar- 
ter of 1938 was selected as the criterion 
variable. An average grade was obtained by 
taking the ratio of points to hours carried. 
For each hour of grade A, four points were 
given; for each hour of B, three points; for 
each hour of C, two points; for each hour of 
D, one point; and for each hour of E 
(failure), no point. Before one can well go 
ahead with the development of new criteria 
for judging academic performance and pro- 
fessional development, those criteria at hand 
should be examined. This study has as its 
problem, the investigation of the relationship 
between a traditional criterion of academic 
success and the various test scores obtained. 
The next step, not here attempted, is to con- 
struct other useful and valid criteria which 
are fairly independent of the course marks. 

The Wherry—Doolittle Test Selection 
Method! has been employed to find the maxi- 
mum multiple correlation with the criterion 
after a correction is made for the chance 
error for each test added. The variables or 
tests are selected in the order of their con- 
tribution to the multiple correlation. The 
first test to be selected is that test which has 
the greatest correlation with the criterion. 
The second is the one which in combination 
with the already selected test gives the great- 
est multiple correlation. The third test is 
that one which combined wih the two se- 
lected raises the multiple correlation the 
most, etc. The increase in the multiple be- 
comes less and less while at the same time 
the chance error increases. The selection may 
continue until the addition of another test 
adds more chance error than actual validity 
or until all tests are included. The Wherry 
shrinkage formula indicates when the point of 


1 Designed by Dr. R. W. Wherry, 
Carolina. 


University of North 
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adding more chance error than validity is 
reached so that no further addition of tests 
is feasible. 

The work is begun on each test at the same 
time and carried out simultaneously with the 
selection of one test at the end of each step. 
This method holds only if no two tests have 
any part in common, which eliminates 
spurious correlations. For instance, if one 
variable were the total of Part I and Part II, 
the variable containing the total may be in- 
cluded if Part I and Part II are excluded 
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from the study; or, if both Part I and fev 
are used separately, the variable containiy 
the total must be excluded. Such exclusion; 
were used at the appropriate points in th. 
calculations. For example, when variable :; 
Part I of the Contemporary Affairs, entere; 
the selected test battery, variable 37, Tot,) 
Score for the Contemporary Affairs, 3; 
dropped from further consideration. 

Table I presents the zero order correlations 
of each of the test scores with the criterion 
These range from 0.012 for variable 9; on 


TABLE I 
CORRELATION OF VARIABLES WITH CRITERION 


Coefficient of 
Variable Correlation 
with Criterion 
A. Interpretation of Data Test 
1. General Accuracy -- - -- a .494 


3. Insufficient Data_______- .318 
4. Accuracy with true and 
false statements_____- . 443 
6. Overcautiousness_-_ . _ _-_- —. 199 
7. Beyond the facts___----- —. 305 
B. — of Beliefs 
9. Liberalism in democracy . 280 
13. Liberalism in nationalism 344 
14. Liberalism in militarism . 361 
15. Total liberalism. _____-_- . 300 
16. Conservatism in 
democracy __........ —, 225 
20. Conservatism in 
nationalism ___-____- ~. 296 
21. Conservatism in 
militarism... _......_- —, 336 
22. Total conservatism’ - -- —. 274 
23. Uncertainty in democracy —.081 
27. Uncertainty in 
nationalism. ______- —.115 
28. Uncertainty in 
militarism___--_-__-_-- —. 088 
29. Total uncertainty _____ ‘ ~—. 070 
30. Consistency in 
democracy _ ____----- . 303 
34. Consistency in 
nationalism Be i aie .318 
35. Consistency in 
militarism ________- : . 361 
36. Total consistency. ____-- . 402 
C. Cooperative Contemporary 
Affairs Test 
St. Fetes Wes... --.cckee-- .317 
38. Political-Social sa . 302 
44. Knowledge of Inter- 
national Events____- : .177 
50. Contemporary Culture . 259 
i} "ia .178 
53. Music and Radio___-__ . 146 
64. Movies___......._.-_- P . 158 


D. Social Problems Test 
59. Number of sound reasons 
checked : . 223 


Coefficient of 
Correlation 
with Criterion 


Variable 


60. Average number of sound 
reasons per conclusion_ 
62. Number of reasons incon- 
sistent with course of 
eae 
66. Number of democratic 
conclusions__________- 
68. Number of compromise 
conclusions ____ ___ = 
70. Defense of special 
priveeege.............. 
73. Defense of individualism 
74. Defense of democratic 
principles_____._____-_- 
75. Defense of compromise 
for sake of expediency _ 


E. Interest Questionnaire 
76. Like Science, Engineer- 
ing, Mathematics_- 
77. Dislike Science, Engi- 
neering, Mathematics - 
78. Like Social Studies_____- 
79. Dislike Social Studies___- 
80. Like English___.____-- 
81. Dislike English. ____-_-- 
86. Like Fine Arts________-- 
87. Dislike Fine Arts_- ---_-- 
94. Like Leadership. __-_-__- 
95. Dislike Leadership - ----- 
96. Like Humanitarian _--__- 
97. Dislike Humanitarian --_- 
98. Like Sociable__________- 
99. Dislike Sociable________- 
100. Like Theoretical_______- 
101. Dislike Theoretical -____- 
102. Like Reading_______-_--- 
103. Dislike Reading- --- ---- 
108. Like Talking. ---------. 
109. Dislike Talking --------- 


F. Intelligence Test 
118. Ohio State University 
Psychological Test - --- 


G. High School Record 
119. Rank in high school 
graduating class_----- 


. 265 














; 1940 | 


humanitarian” of the Interest Question- 
« to 0.007 for variable 118 on the Ohio 
te University Psychological Examination. 
her than the Ohio State University Psy- 
Examination, the Interpretation of 
t seems, in general, to have the great- 
lidity, for predicting academic success. 
'y be “somewhat to be expected” 
he test apparently reflects many of the 

; emanded in class work. 
fable II indicates the variables selected by 


Wherry—Doolittle Method in the order 
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ing quarter. If such a figure be taken as an 
estimate of the reliability of the criterion, 
then the battery of variables selected ap- 
proaches the limit of correlation with the 
criterion. 

The multiple correlation of 0.692 indicates 
that approximately one half of the variance 
for predicting academic success has been ex- 
plained. The addition of the last test ac- 
counted for less than 0.ooo1 of a perfect re- 
lation between the criterion and the tests pre- 


TABLE II 


BeTA AND b WEIGHTS, CORRELATION AND CUMULATIVE MULTIPLE CORRELATION 
COEFFICIENTS FoR EACH SELECTED VARIABLE 


Selected Variable 


Ohio State University 
Psychological Test_ - 
1. General Accuracy 
Interpretation of Data Test 
119. Rank in High School 
Graduating Class- 
6. Total Consistency 
Seale of Beliefs) - 
87. Dislike Fine Arts 
Interest Questionnaire) - _- 
8. Political-Social 
Cooperative Contemporary Affairs 
Average Number of Sound Reasons per 
Conclusion (Social Problems Test 
Contemporary Culture 
Cooperative Contemporary Affairs 


i their selection. This table also shows the 
multiple correlation coefficient (corrected for 
chance error) at the stage represented by the 
inclusion of that test in the battery. For 
example, the addition of the second test, vari- 
able r on General Accuracy (Interpretation 
of Data Test) raised the correlation with the 
criterion from 0.607 to 0.649. The addition 





dicting it. No further test additions are 
Cumulative 
Coefficient of Coefficient of 


Beta b Weight Correlation Multiple 
Weight with Criterion Correlation 

. 3825 . 009 607 . 607 

. 198 . 009 . 494 649 

. 181 . 096 .441 . 668 
120 . 008 402 . 678 
114 .031 . 189 . 684 

ani . 005 . 302 . 689 
070 004 . 265 . 692 

. 041 ~. 002 . 259 . 692 


feasible since the tests are selected in order 
of their contribution to the multiple. 

The Table also shows the Beta or net 
regression coefficients and the b or gross score 
regression coefficients. The Beta weights are 
used to predict the individual’s criterion 
score only when his test scores have been 
transformed into standard scores. The regres- 
sion equation is 


Z. == .325X1,— + .198X, + .181X,,9 + .12OXgg — .II4Xg7 ++ .TIIXy, + .070X,,, — .O4I X59 





of a third variable, 119, Rank in High 
School Graduating Class, raised the multiple 
correlation to 0.668, etc. All the eight vari- 
ibles selected produced a multiple correlation 
0! 0.692 (corrected for chance error) with 
point-hour ratio. This is about the same mag- 
nitude as the correlation of the point-hour 
ratio of one quarter with that of the succeed- 


By Beta weights only are contributions of 
the various tests on a comparable basis. They 
tell us what part of a standard score should 
be included to give the most probable predic- 
tion. Thus by examination of the Beta 
weights we learn the relative importance of 
each selected test toward the purpose in 
question. 
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TABLE III 
INTERCORRELATION OF SELECTED VARIABLES 
V V \ V \ \ 
Selected Variable 118 1 119 36 37 8 60 
118. Ohio State University 
Psychological Test 477 . 448 . 381 179 . 320 . 308 415 
1. General Accuracy 
(Interpretation of Data A477 315 . 400 025 . 230 . 205 
119. Rank in High School c 
Graduating Class . 448 315 . 222 . 055 .141 164 ‘ 
36. Total Consistency 
(Seale of Beliefs . 381 400 222 ~. 066 . 224 174 4 
87. Dislike Fine Arts 
(Interest Questionnaire -.179 025 055 . 066 .011 023 
38. Political-Social 
(Contemporary Affairs . 320 230 .141 . 224 O11 114 48 
60. Average Number of 
Sound Reasons Per 
Conclusion 
(Social Problems Test) - . 308 205 164 .174 . 023 .114 
50. Contemporary Culture 
(Contemporary Affairs) 418 .178 . 194 . 146 . 135 , 481 107 
In the practical problem of setting up pre- 36. Total Consistency (Scale of Beliefs 
dicted scores it is more convenient to use 87. Dislike Fine Arts (Interest Question- 
gross scores regression coefficients. These are naire) 
usually referred to as b weights. With b 38. Political-Social (Contemporary Af. 
weights the regression becomes fairs Test) 
X, 0o9gX,,, + .009X, + .096X,,, + .008X,, — .031X,, 
+- .005X,, + .004X,,, — .002X,, + K 
The intercorrelations of the selected vari- 60. Average Number of Sound Reasons 
ables are shown in Table ITI. per Conclusion (Social Problems 
¢ Test) 
SUMMARY . 
50. Contemporary Culture (Contempo- 


In the comparison of the Beta weights we 
find that the Ohio State University Psy- 
chological Examination has the largest. It is 
interesting to note that Dislike for Fine Arts 
and Contemporary Culture have negative 


weights. 
The prediction battery is 
118. Ohio State University Psychological 
Test 
1. General Accuracy (Interpretation of 
Data Test) 
119. Rank in High School Graduating 


Class 


rary Affairs Test) 


It is significant to note that scores from 
each of the tests used appear among the se- 
lected tests. This perhaps attests the excel- 
lence of the @ priori selection of tests to be 
used in this program. The significant pre- 
dictors are, with the exception of Dislike for 
Fine Arts, scores which reflect abilities, in- 
formation or consistency of response, rather 
than a point of view such as liberalism 
nationalism, or conservatism. 











PERFORMANCE IN THE IOWA 


QUALIFYING EXAMINATION 


OF MAJORS IN VARIOUS ACADEMIC DEPARTMENTS 
WITH IMPLICATIONS FOR COUNSELING 


Dewey B. Stuit and Mary CAarRrRoL_t DONNELLY 
State University of lowa 


The choice of a major is a problem which 
‘roubles a rather large number of college stu- 
dents. Because of superficial interest in a sub- 
‘ect, admiration for a certain instructor, or 
the suggestion of a friend, a student may 
make a choice which he will regret later in 
his educational career. Many students do not 
have a satisfactory opportunity to evaluate 
their achievements in an objective way and 
through that means arrive at a sound judg- 
ment concerning their field of concentration. 
it has been suggested by many writers that 
aptitude and achievement tests should provide 
the student with the necessary objective facts. 
However, the vast majority of studies in 
which such tests have been used have not 
oroved exceedingly helpful because the results 
wre confined to correlations showing the rela- 
tionship between a predictive index and a 
criterion. Generally the differential prediction 
value of various measuring instruments has 
not been ascertained. 

The purpose of the present study is to 
investigate the possibility of utilizing test 
results in that area of counseling which in- 
volves the student’s choice of a major subject. 
Specifically, it seeks to determine whether 
there are any differential characteristics of 
majors in various academic departments 
which are revealed by their performance in 
the Iowa Qualifying Examination. This ex- 
‘mination, consisting of the Iowa High School 
Content Examination, the Mathematics Apti- 
tude Test and English Training Test of the 
lowa Placement Examinations, and the Iowa 
Silent Reading Test, is administered to all 
‘reshmen when they enroll in the University. 
The purpose of this examination is to provide 
some indication of the student’s scholastic 
iptitude and to assist the counselor in his 
educational advisory work with students, par- 
ticularly at registration time. The present 
‘tudy investigates the initial test performance 
' students who later succeed in obtaining 
wademic degrees in various departments. If 
ispection reveals that one may partially de- 
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scribe the successful students in various aca- 
demic departments in terms of their variabil- 
ity of achievement in the different tests of the 
Iowa Qualifying Examination, such informa- 
tion should afford valuable supplementary 
material in counseling a student concerning 
his choice of a major subject. If it is found 
that there are no such differential character- 
istics, then the use of the test profile, as such, 
is of little value when counseling in this area. 
In brief, the questions which this study seeks 
to answer are as follows: 


(1) With respect to initial test scores, is 
there a characteristic test profile for stu- 
dents who have successfully received aca- 
demic degrees in a given department? 

(2) Is there a characteristic lower limit 
suggestive of a critical score for majors in 
various academic departments? 


Nine major departments in the College of 
Liberal Arts at the State University of Iowa 
were selected for study. The selection in- 


cluded: (1) Mathematics; (2) English; 
(3) Journalism; (4) Political Science: 
(5) History; (6) Fine Arts: Music and 


Graphic Arts; (7) Physical Science: Chem- 
istry and Physics; (8) Biological Science: 
Biology and Botany; (9) General Science. 
These groups were selected in an attempt to 
represent all of the major fields of concentra- 
tion, combining those fields assumed to be 
closely allied. 

In order to secure an adequate sample of 
students in each of these departments, it was 
necessary to obtain data from classes gradu- 
ated in June Convocations at the State Uni- 
versity of lowa for the years 1935-1938 in- 
clusive. All students graduated during that 
period who had taken the Iowa Qualifying 
Examination upon entrance to the University 
were included as subjects for the study. This 
eliminated students who had transferred to 
the University from other institutions and 
those who had entered during the summer 
terms. The total group included 365 students. 
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Scores in the individual examinations and the 
composite, consisting of the sum of the raw 
scores of the four individual examinations, 
were converted to percentile ratings. 

The results of this study provide a quanti- 
tative description of the performance of the 
nine major groups in the individual examina- 
tions and composite score of the Iowa Qual- 
ifying Examination. Detailed findings are 
summarized in Tables I and II. 

Table I shows the means and standard 
deviations of the major groups in each of the 
individual examinations and composite score 
of the Iowa Qualifying Examination. It is 
interesting to note that the two highest rank- 
ing groups in terms of composite score, Math- 
ematics and English, are also the least vari- 
able. The third ranking group, Physical 
Science, presents an interesting contrast in 
that it appears more variable than all other 
groups except the Fine Arts Majors, who 
show the widest variability of any group. The 
lowest ranking groups, Biological Science, 
Fine Arts, and Political Science all show wide 
variability. 

The examinations in which each of the 
major groups receives its highest and lowest 
rankings constitute interesting observational 
data. Mathematics, Physical Science, and 
General Science Majors show their highest 
mean score in the Mathematics Aptitude 
Examination; English and Fine Arts Majors 
in the English Training Examination: Bio- 
logical Science, Political Science, and History 
Majors in the High School Content Exam- 
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ination; and Journalism Majors in the [oy- 
Silent Reading Test. 

With respect to low scores, it will be notes 
that the Mathematics Majors earned th: 
lowest mean score in the Iowa Silent Res 
Test; Physical Science, Genera! Scien: 
Biological Science, and Political Scie 
Majors in the English Training Exa 
English and History Majors in the \ort 
ematics Aptitude Examination and ], 
and Fine Arts Majors in the Matly 
Aptitude and High School Content Ex 
ination. 

An additional representation of \ 
in performance is given in Table I] 
shows the decile ratings of all major 
in the individual examinations and c 
score of the Iowa Qualifying Examinatio: 
per cent of cases. With regard to th 
limits, it will be noted that the groups shoy 
ing highest per cents in the roth deci 
the following: Mathematics Majors in th 
Mathematics Aptitude Examination, Matb- 
ematics Majors in composite score, \Matb- 
ematics Majors in High School Content Fy. 
amination, English Majors in English Traip- 
ing Examination and English Majors in « 
posite score. In these instances, 40° or more 
of the cases fall within the roth decili 

Among the groups showing the lowest per 
cents in the roth decile are History Major 
in the English Training Examination, Jour- 
nalism Majors in the High Schoo! Content 
Examination, Biological Science Majors in 
the Iowa Silent Reading Test and Fng'ish 


TABLE I 


MEAN PERCENTILES AND STANDARD DEVIATIONS OF THE MAJOR GROUPS IN EACH OF THE IND! 
VIDUAL EXAMINATIONS AND COMPOSITE SCORE OF THE IOWA QUALIFYING EXAMINATION 


H.S.C. 
Major N M o 
Mathematics ..._._..._._._.___- 21 82 20.0 
English ...... paccntae 77 18.2 
Phys. Science ____-_-- — - 72 27.0 
Gen. Science ___~ * on 67 28.1 
Journalism ___ ~~ cubeienae 61 25.3 
History ; casts aa 67 23.4 
Pol. Science __ eee 64 28.0 
Fine Arts Saebeiiicdstisigpanektia caked GE 54 30.0 
Biol. Science _________ — 62 26.5 


1.S.R.* M.A. E.T. CS 
M o M og M go M 0 
73 22.6 93 5.3 75 25.8 86 21.6 
77 22.0 66 25.3 82 18.1 78 212 
67 27.1 76 25.0 66 30.0 73 284 
61 25.2 71 23.7 55 26.8 69 23.6 
68 25.1 61 26.9 67 23.5 66 23.5 
63 30.3 55 29.9 62 25.2 63 284 
59 26.3 59 28.7 56 27.1 61 27.) 
60 22.4 54 31.4 70 26.3 60 30.7 
60 26.9 59 26.2 55 24.2 59 27.8 


The abbreviations are to be interpreted as follows: 


H.S.C. = High School Content Exami- 


nation 
1.S.R. = lowa Silent Reading Test 


M.A. = Mathematics Aptitude Exam 

ination 
E.T.= English Training Examination 
C.S. = Composite Score 


* Iowa Silent Reading Test Scores were not available for one English, one Political Science, 
and three Fine Arts Majors. 
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TABLE II 


Pe \TINGS OF ALL MAJOR Groups IN INDIVIDUAL EXAMINATIONS AND COMPOSITE SCORE OF 
THE IOWA QUALIFYING EXAMINATION IN PER CENT OF CASES 








1 2 3 4 5 6 7 8 9 10 N 
“HSC a 48 143 48 19.0 524. 21 
LSI . 4.8 4.8 4.8 48 143 148 286 23.8 21 
yA 48 238 714. 21 
ET. See 14.3 48 148 19.0 428 21 
a 4.8 4.8 4.8 19.0 66.7 21 
g | 3.4 1.7 11.9 3.4 as 2126.. 322 32.2 59 
-y 1.7 1.7 6.8 10.2 10.2 8.5 23.7 35.6 8 
\ - 5.1 8.5 8.5 1.7 11.9 6.8 15.3 25.4 169 59 
‘ie 1.7 6.8 6.8 10.2 8.5 18.6 47.5 9 
1.7 5.1 13.6 11.9 8.5 18.6 40.7 59 

Phys. Se. 
H.S.C. 7 9.0 4.5 4.5 18.2 1.5 4.5 18.2 36.4 22 
TS 4.5 13.6 18.2 13.6 9.0 45 31.8 22 
M.A. a 9.0 4.5 9.0 4.5 9.0 27.3 36.4 22 
E.T. 4.5 13.6 13.6 4.5 4.5 18.2 9.0 31.8 29 
Cs 9.0 9.0 9.0 9.0 9.0 182 364 22 

Se 
HS. 7.4 7.4 11.1 7.4 7.4 74 25.9 259 27 
LS.R. 3.7 11.1 18.5 22.2 3.7 18.5 3.7 18.5 27 
M.A. c 3.7 7.4 18.5 148 111 1211.1 383838 27 
|, anes 3.7 7 a7 wt i835 213 Ms i113 3.7 148 27 
Cs. ee 7.4 97 185 i111 148 185 222 2&1 
LS.C See 5.4 8.9 10.7 8.9 10.7 8.9 16.0 19.6 10.7 56 
LS.R 5 i. ae 3.6 3.6 73 5.4 89 143 10.7 286 160 56 
MA : a aa 5.4 5.4 1438 7.1 5.4 19.6 10.7 10.7 19.6 56 
¢ vena 3.6 7.1 16.0 125 143 89 12.5 25.0 6 
A SEE 8.9 12.5 8.9 1.7 160 143 17.9 19.6 56 
H.S.C. eae 5.7 2.9 5.7 5.7 148 171 #2114 «17.1 20.0 85 
i eee 2.9 8.6 2.9 5.7 11.4 5.7 11.4 148 17.1 200 35 
M.A. ae 5.7 1438 1438 5.7 5.7 11.4 29 229 143 35 
E.T. sisted 5.7 5.7 11.4 5.7 5.7 20.0 20.0 17.1 8.6 35 
C.S. elie acer 5.7 5.7 5.7 14.3 20.0 11.4 114 229 35 

Pol. Se. 
H.S.C. — 5.3 6.7 8.0 4.0 27 14.7 18.7 16.0 20.0 75 
RRR, 2.7 5.3 5.3 16.0 2.77 160 183 138.3 9.3 14.7 74 
RE oe 5.3 4.0 9.3 8.0 13.3 5.3 10.7 12.0 14.7 17.8 175 
|. Saas e 1.3 6.7 8.0 14.7 12.0 93 10.7 12.0 10.7 18.3 75 
i EER TS 4.0 6.7 5.3 6.6 10.7 8.0 18.3 160 1838 160 75 

F. Arts 
H.S.C cae ee 1 64 1 1 1 47 
[' poe 1 18.2 1 44 
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Training Examination and Political Science 
Majors in the English Training Examination. 
In these instances, 149% or less of the cases 
fall within the roth decile. It is noticeable 
that the examination in which these lower 
scoring groups most frequently fail to score 
high is the English Training Examination. 
Fine Arts Majors are an exception, earning 
their highest score in the English Training 
Examination. 

Probably the chief interest in this table lies 
in the possibility of establishing lower limits 
or “critical scores” for the major groups in 
the individual examinations. It is recognized 
that the establishment of such “critical 
scores” is an arbitrary matter. While it is 
impossible to say that cases rating lower in 
the examinations than the groups studied 
would not have succeeded, it is possible to 
express quantitatively the relative achieve- 
ment of successful groups in this study. For 
example, the most striking observation is that 
with respect to Mathematics Majors, less than 
five per cent fall below the ninth decile in the 
Mathematics Aptitude Examination. In the 
case of English Majors, only slightly more 
than 25% fall below the eighth decile in the 
English Training Examination; less than two 
per cent fall below the fifth decile. 

Such data as these may be contrasted to 
that of the Fine Arts Majors, for which on 
three of four tests, approximately 45% fall 
within and below the fifth decile. In a similar 
manner, by adding up from the bottom one 
can determine the per cent of cases who fall 
at or below a certain decile rating in each of 
the examinations. For example, if one were 
interested in determining the percentage of 
cases of Mathematics Majors falling below 
the fiftieth percentile in any examination this 
could be done by adding all the percentages 
falling below the sixth decile for that exam- 
ination in Table II. 

In order to make a more careful statistical 
study of the results, analysis of variance was 
employed in studying the significance of dif- 


1 All of the actual computations are not reproduced here 
because of limitations of space. The means, standard devia- 
tions and differences in means can be obtained from Table I 
\ discussion of the statistical technique known as Analysis of 
Variance may be found in: 

Fisher. R. A.: Statistical Methods of Research Workers. 
Sixth Edition, Oliver and Boyd, Edinburgh, 1936, pp. 214-290. 

Lindquist, E. F.: Statistical Analysis in Educational Re- 
search. Houghton—Mifflin Company, New York, 1940. In 
Publication 

Snedecor, G. W.: Statistical Methods, Collegiate Press, Inc., 
Ames, Iowa, 1938, pp. 179-247. 

Yule. G. U.; a Kendall, M. G.: An Introduction to the 


Theory of Statistics. Eleventh Edition, Charles Griffin and 
Co., London, 1937, pp. 444-458. 
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TABLE III 
RESULTS OF ANALYSIS OF VARIANCE Witury 


TESTS AND BETWEEN TEST MEANS 


VARIOUS MAJor GROUPS 


d.f. 

. Mathematics Majors 
Between tests... 4 
Within tests_.__ 100 
ee 104 
F Ratio 3.53* 

. English Majors 
Between tests__ 4 
Within tests... 289 
EEE 293 


F Ratio 4.78** 


Between tests__ 4 
Within tests___ 105 
Total ________ 109 
. General Science Majors 
Between tests__ 4 
Within tests_.__ 130 
. | aes 134 


F Ratio 1.76 


. Journalism Majors 


Between tests__ 4 
Within tests___ 275 
.. 


F Ratio 1.58 


. History Majors 


Between tests__ 4 
Within tests_.__ 170 
a 174 

. Political Science Majors 
Between tests... 4 
Within tests_.__ 369 
a 373 

. Fine Arts Majors 
Between tests... 4 
Within tests... 227 
CO aa 231 
F Ratio 2.30 


Between tests... 4 
Within tests___ 110 


Total 


Sum of 
Squares 


5611.4 
39769.5 


8187.4 
123636.5 


. Physical Science Majors 


1599.5 
75033.0 


4443.4 
81638.5 


2621.1 
113733.1 


2602.5 
121831.3 


2428.0 
27482.8 


7891.1 
194530.0 


. Biological Science Majors 


570.66 
80042.53 


5, V 


FOR THE 


Variance 


14028 


207 ¢ 


8998 


1110.8 
627.9 


607.0 


744 


1972.7 


R569 


142.66 


m9" 65 
le 


* To be significant at the 1% level “F” must 
equal 3.51 and at the 5% level 2.46. 


** To be significant at t 


he 1% level “F” 


must equal 3.38 and at the 5% level 2.40. 








TABLE IV 


or DIFFERENCES IN MEANS AND 


SUMMARY 
" Tests OF THEIR STATISTICAL SIGNIFICANCE 
MATHEMATICS AND ENGLISH MAJORS 


FOR 
BETW! en ALL INDIVIDUAL EXAMINATIONS 
anp COMPOSITE SCORE IN THE IoWA QUALI- 


FYING EXAMINATIONS 


Part 1a. Mathematics Majors 


Examinations* Diff. ¢ diff. ee 
eT) ae 4.5 2.44 
M.A.-LS.R. actnemenioeaiione ae 5.1 3.92 
eR) 18 5.7 3.16 
ca.58@ ............18 68 iff 
CE-B.T. ...-n0<= ee 7.3 1.51 

Part 1b. English Majors 

US.C-M.A. ssctoubeemeceuael aa 4.2 2.62 
LS.R.-M.A. --------- 11 4.5 2.44 
2 ) a _ 16 4.2 3.81 
CRATES, cccncnccnnen 3B 4.4 2.73 
*The examination with the highest mean 


nercentile score is listed first. 

** For these two groups “t” must exceed 
258 to be significant at the 1% level and 1.96 
at the 5% level. 

ference in means.* The variance of each 

major group in all of the individual exam- 

inations and composite score was analyzed. 

By applying the “F” test, the results of which 

are presented in Table III, it was found that 

no significant** differences in means existed 
within any of the groups except for the Math- 
ematics and English Majors. These differ- 
ences and tests of their significance are pre- 
sented in Table IV. The results indicate that 
_ the Mathematics Majors perform significantly 
higher in the Mathematics Aptitude Exam- 
' ination than in the Iowa Silent Reading Test 
_ and English Training Examination. The data 
also indicate that the English Majors score 
_ significantly higher in the High School Con- 
tent Examination, English Training Exam- 
ination and composite score than in the 

Mathematics Aptitude Examination. Since 

there are no significant differences within any 

i the other major groups it does not appear 

advisable for a counselor to rely heavily upon 

the pattern of scores in the individual exam- 
nations for differential prediction. 

The variance of all major groups in each 
it the individual examinations was also ana- 

lyzed. The results are reported in Table V 


“One of the underlying assumptions in analysis of vari- 
ince is normality of the population from which statistics 
such as the mean are aubeal ont hence the use of percentiles 
* not fully justified. It is believed, however, that the result- 
ag errors will not be large. 

_"* Because of the nature of the data the writers have 
‘nosen the 1% level as their criterion of statistical signifi- 
— 4 evaluating all of the differences in means found in 


this study, 
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TABLE V 


RESULTS OF ANALYSIS OF VARIANCE WITHIN 
GrRoUPS AND BETWEEN GROUP MEANS FOR 
EacuH EXAMINATION AND THE COMPOSITE 
ScoRE 


Sum of 
d.f. Squares Variance 


1. High School Content Examination 








Between groups 8 21390.8 2673.8 
Within groups. 356 240163.7 674.6 
succeeds 364 
F Ratio 3.96* 

2. Iowa Silent Reading Test 
Between groups 8  15293.5 1911.6 
Within groups. 351 223859.9 637.7 
a 359 
F ratio 2.99 

3. Mathematics Aptitude Examination 
Between groups 8 32324.4 4040.5 
Within groups. 356 256496.8 720.4 
TN Seceaici sults 364 
F ratio 5.60 

4. English Training Examination 
Between groups 8  31042.0 3880.2 
Within groups. 356 209817.2 589.3 
TO nicnein 364 
F ratio 6.58 

5. Composite Score 
Between groups 8  23500.4 2937.5 
Within groups. 356 224692.0 631.1 


Total 
F ratio 4.65 


*In this table “F” must equal 2.56 to be 
oo at the 1% level and 1.96 at the 5% 
evel. 


and reveal that within every examination 
there are statistically significant differences 
in means for some of the major groups. The 
differences found to be significant or ap- 
proaching significance are presented in Table 
VI. The mean scores of Mathematics Majors 
are significantly higher than those of other 
major groups in the following examinations: 
Mathematics Aptitude, higher than all other 
major groups; High School Content, higher 
than Biological Science, Journalism, Political 
Science, and Fine Arts Majors; English 
Training, higher than Biological Science, 
General Science and Political Science Majors; 
Composite score, higher than all major groups 
except English and Physical Science. 
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English Majors exhibit significantly higher 
mean scores than other groups in the follow- 
ing examinations: English Training, higher 
than all major groups except Physical Science 
and Mathematics: lowa Silent Reading Test, 
higher than Biological Science, General Sci- 
ence, Political Science and Fine Arts Majors; 
High School Content Examination, higher 
than Journalism, Political Science and Fine 
Arts Majors; Composite score, higher than 
Biological Science, Journalism, Political Sci- 
ence, History and Fine Arts Majors. 


TABLE VI 
SUMMARY OF DIFFERENCES IN MEANS AND 
Tes7s OF THEIR SIGNIFICANCE FOR ALL 
Masor Groups WITHIN EACH INDIVIDUAL 


EXAMINATION AND COMPOSITE SCORE OF THE 
IowA QUALIFYING EXAMINATION 


Part la. High School Content Examination 


Maior Groups* Di ediff. “t’’** 
Math.—Biol. Se. 20 7.0 2.86 
Math.—Gen. Sc. 15 7.0 2.14 
Math.—Journ. 21 5.5 3.82 
Math.—Pol. Se. ——_ 18 5.4 3.33 
Math.—Hist. 15 5.9 2.54 
Math.—Fine Arts 28 6.2 4.52 
Phys. Se.—Fine Arts 18 7.2 2.50 
Eng.—Biol. Sc. 15 6.0 2.50 
Eng.—Journ. 16 4.1 3.90 
Eng.—Pol. Se. —_- 13 4.0 3.25 
Eng.-Hist. 10 4.6 2.17 
Eng.—Fine Arts 23 5.0 4.60 
Hist.-Fine Arts ‘aeeuas aee 5.9 2.20 
Part 1b. Iowa Silent Reading Test 
Math.—Pol. Se. a 5.8 2.41 
Math.—Fine Arts . 18 6.0 2.17 
Eng.—Biol. Se. - ioe EE 6.3 2.70 
Eng.—Gen. Sc. — 16 5.6 2.86 
Major Groups* Dif. «dif. “E"e* 
Eng.—Pol. Se. —-. com BO 4.2 4.29 
Ea 14 5.9 2.37 
Eng.-Fine Arts ____---. 17 4.4 3.86 
Part le. Mathematics Aptitude Examination 
Math.—Phys. Sc. ___---- 17 5.4 3.15 
Math.-Biol. Sc. _.__.--_. 34 5.6 6.07 
Math.-—Gen. Se. ._..-.-. 22 4.7 4.68 
ES ee | 3.6 7.50 
Math.—Journ. _..._._.____. 32 3.8 8.42 
Math.—Pol. Sc. .._..___- 34 3.5 9.71 
Math.—Hist. ........... 38 5.2 7.31 
Math.—Fine Arts _____~ 39 4.7 8.30 
Phys. Sc.—Biol. Se. _____ 17 7.6 2.24 
Phys. Se.—Journ. —~____- 15 6.4 2.34 
Phys. Se.—Pol. Se. —___- 17 6.3 2.70 
Phys. Sc.—Hist. ._._..._. 21 7.3 2.88 
Phys. Se.—Fine Arts __.. 22 7.0 3.14 
Gen. Sc.—Hist. _._______ 16 6.7 2.39 
Gen. Se.—Fine Arts ____ 17 6.4 2.66 
Eng.—Fine Arts ________ 12 5.7 2.11 
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Part 1d. English Training Examinat 


Major Groups* Diff. o diff. 
Math.-Biol. Se. ________ 20 7.6 9 
Math.—Gen. Se. _____- 20 7.6 ? 
Math.-Pol. Se. ___.---_ 19 6.4 29° 
Eng.—Phys. Sc. -------- 16 6.7 
Eng-moe. Se. _......... 37 5. 

Bee =s0G8h, ........... 15 a! 
Eng.—Gen. Sc. ~___-- 27 7 
Eng.—Pol. Se. .----- . 26 3.9 
Eng.-Hist. _______- 20 1.9 
Eng.—Fine Arts ____- 12 1.6 

Fine Arts—Biol. Sc. - 15 6.3 2 
Fine Arts—Gen. Se. ___- 15 6.4 ) 
Fine Arts—Pol. Sc. _____ 14 1.9 9 & 
Journ.—Biol. Se. ___---. 12 6.0 2 
sourn.-Fol. Se. ........ 1 1.4 2 


Part le. Composite Score in Qualifying 


Examination 
Math.-Biol. Se. —- 7.4 
Math.—Gen. Sc. — i7 6, 2 49 
Math.—Journ. __- oien ae 7 
Math.—Pol. Se. —_- 25 
Major Groups* Diff. o¢ diff 
Math.-Hist. _____- siiapasigs a 6.8 
Math.—Fine Arts _______ 26 6 
Eng.—Biol. Sc. __._______ 19 6.3 
Eng.—Journ. .._...-_-.. 12 1.2 2 
Eng.—Pol. Se. ...------- 17 1.2 
ne |: 5.5 27 
Eng.—Fine Arts ________ 18 5.3 { 


*The major group with the highest mea 
percentile score is listed first. 

** For all these groups “t” must exceed 2.58 
to be significant at the 1% level and 1.96 a 
the 5% level. 


Physical Science Majors show a signil- 
cantly higher mean in the Mathematics Apt- 
tude Examination than Political Scienc: 
History and Fine Arts Majors. General S¢- 
ence Majors exhibit a mean score in Matb- 
ematics Aptitude which is significantly higher 
than that of the Fine Arts Majors. The mean 
score of Fine Arts Majors is significant) 
higher in English Training that that of the 
Political Science Majors. Several other difier- 
ences are quite large but fall slightly below 
the magnitude required for significance at the 
one per cent level. 

Some interesting and helpful observation 
may be made concerning these significant di 
ferences. The highest ranking groups in con 
posite score, namely, Mathematics and Enz 
lish Majors, are most frequently significant!) 
higher than other groups in the individu! 
examinations. The third ranking group, Phys 
ical Science Majors, is also third in frequency 
of significantly high scores in the individvsl 
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examinations. It is worthy of note that Eng- 
sh Majors are significantly higher in Com- 
site score and High School Content Exam- 

than Journalism Majors. Fine Arts 
while characteristically lower than 
‘other groups in most instances, score sig- 
nily higher than Political Science Majors 


dal 


the English Training Examination. In this 
nnection it should be pointed out that high 
aptitude is probably not as impor- 

factor in determining success in Fine 

\rts as in the other academic areas. Tests 
; those investigated in this study should 
ertainly be supplemented with others 
counseling students concerning their prob- 


success in art. 


CONCLUSIONS 
basis of the results of this study it 


On the 


yppears that a knowledge of the distribution 
of successful majors in various aca- 


res 
( 
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demic departments should prove helpful in 
counseling students concerning their choice of 
a major subject. Students ranking high in 
the Iowa Qualifying Examination may be 
afforded wide choices of major subjects; 
however, the choice of those students ranking 
low, if they wish to compete with students of 
their own abilities, is more restricted. As this 
study dealt only with students who had suc- 
cessfully completed work in various academic 
departments, no absolute lower limits of per- 
formance in the Iowa Qualifying Examination 
can definitely be established. However, if one 
were willing to assume that the level of per- 
formance represented by students included in 
this study is indicative of the level of ability 
required for success in each of the major 
fields, then the results of this investigation 
should be of some value in counseling stu- 
dents with regard to their academic majors. 








THE RELATIONSHIP BETWEEN SCORES ON THE CaAyp 
INTELLIGENCE SCALE AND SUCCESS IN GRADUATE WORK 
AT COLORADO STATE COLLEGE OF EDUCATION 


LORAINE BRUCE 


Pampa Senior High School 
Pampa, Texas 


1. PROBLEM 


The object of this study was to determine 
the validity of the I.E.R. Intelligence Scale 
CAVD, Levels M, N, O, P and Q for the pur- 
pose of predicting achievement at the master’s 
level in the Colorado State College of Educa- 
tion. The CAVD Scale is made up of four 
series of tasks: completions, arithmetical 
problems, vocabulary and directions. The 
upper five levels are for use in higher under- 
graduate and graduate divisions of the col- 
lege. 


SUBJECTS AND TESTS 


All students matriculating on the master’s 
level from June, 1936 to August, 1938 were 
given the I.E.R. Intelligence Scale CAVD, 
Levels M, N, O, P and Q, Form 4, and the 
American Council on Education Cooperative 
English Test. Some of these students took the 
battery of twelve education tests given to 
those who major in education and who have 
elected the four quarter plan for the master’s 
degree. Most of the students on whom the 
A.B. degree was conferred by Colorado State 
College of Education had taken the American 
Council on Education Psychological Exam- 
ination while they were undergraduates. 


3. RESULTS 


Of the students on the master’s level who 
took the CAVD, 440 were in the graduate 
school at least two quarters and took a min- 
imum of twelve quarter-hours each quarter. 
The majority of this graduate work was done 
during the summer quarters. In order to com- 
pute the reliability of teachers’ marks and 
other correlations in which teachers’ marks 
were involved, the point-hour ratio was found. 
The point-hour ratio was computed by divid- 
ing the total number of points by the total 
number of hours. The points were found by 
multiplying the number of each course by the 
point values of the letter grades. The point 
value of “A” is 5; of “B” is 4; of “C”, 3; 
of “D”, and of “F”’, 1. The reliability of 


*° 
@y 


teachers’ marks as found by computing the 
product-moment coefficient of correlation be. 
tween the point-hour ratios for the first quar. 
ter and the second quarter for this group o/ 
440 students is .467 + .025. This is a rel. 
atively low reliability coefficient. The 
product-moment coefficient of correlation be. 
tween scaled scores on the CAVD Intelligence 
Scale and the total point-hour ratio 
first two quarters of graduate work wa 
.310 + .029. Although this is a significant 
correlation, yet it is low if the CAVD jst 
be used as a predictive measure for teachers 
marks. 

The means of the point-hour ratios com- 
puted from the first quarter of twelve 
more hours were found for two groups; fi 
the group making below the 25th percentile 
and second, the group making scores above 
the 75th percentile on the CAVD. The mean 
of the first group (V = 243) was 3.72 and 
the mean of the second group (.V 


OL the 


ret 
Ts 


142 
was 4.25 and the difference between thes 
means was .53. Since the probable error o! 
the difference was .037, there is virtual cer- 
tainty that there is a significant differenc: 
in fact, using the ratio, D/PE, the chances 
are better than 99.99 in 100 that this is 2 
true difference. 

There were 134 students on the masters 
level who took both the CAVD Intelligence 
Scale and the American Council on Education 
Psychological Examination and who had 
taken at least twelve quarter-hours in 
graduate school. A product-moment coelt 
cient of correlation of .752 = .025 was loune 
between scaled scores on the CAVD Inte’ 
gence Scale and sigma scores on the America 
Council Psychological Examination. Perce 
tiles on the American Council Psychologic# 
Examination were translated into sigma sor 
by using a distribution truncated at 
In order to avoid negative o values, the 2" 
point was put at the lower end of the sc 
which in this case was at —5 a. The corres 
tion is significant and indicates a relationship 
53 per cent better than pure chance. When 3 
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Waren, 494 


mere chance agreement between two series is 
1—r 


regarded as zero then is the per- 


; 2 
centage of disagreement and 1— ¥/ : — af 
the percentage of agreement between the two 
cries. Using the same group of students, the 
efficient of correlation was .232 + .055 be- 
rween scaled scores on the CAVD Scale and 
total point-hour ratio of marks on the grad- 
yate level. This indicates only a slight corre- 
lation. A third coefficient of .304 + .053 was 
found between sigma scores on the American 
Council Psychological Examination and the 
total point-hour ratio of marks on the mas- 
ter’s level. This is a slightly better correlation 
than the preceding one. The coefficient of 
222 indicates a correlation 31 per cent better 
than chance and that of .304 a correlation 33 
ner cent better than chance. 

- The difference between the two preceding 
correlations (.23 and .30) is .o7 and the 
probable error of the difference is .o41. In 
der to find out whether this is a significant 
difference, the following formula* for the 


probable error of the difference between two 
coefficients of correlation was used: 
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In order to complete matriculation, mas- 
ters’ students were required to take the 
American Council Cooperative English Test; 
the parts given were English Usage and Spell- 
ing. The product-moment correlation (NV == 
748) between scaled scores on the CAVD 
Intelligence Scale and scaled scores on the 
English Usage Test was .553 + .017; that 
between scaled scores on the CAVD and 
spelling .502 + .o19. These r’s show sub- 
stantial relationships but they also indicate 
that different abilities were measured. 

Until the summer of 1936, there was only 
one method, the thesis plan, of working for 
a master’s degree in Colorado State College 
of Education. Beginning then, graduate stu- 
dents were given a choice of two programs of 
work leading to the master’s degree. The 
thesis plan was one of the two programs; the 
second method was known as the examination 
plan. One of the three required comprehen- 
sive examination programs covered the gen- 
eral field of education. In the summer of 
1937, the thesis pian was called the A plan 
and the second plan was changed to the B 
or four quarter plan. Those who had started 
the examination plan in 1936 and who wished 
to continue were allowed to do so. 





PE, = 
in which 


13 2 : 121 


": V1o%13(1 — P23 — 


/ PE’, +PE*, —2r, , PE; PE; 
1 13 3 12 13 


Vio — P13 2% P17 03) 





2(1—143) (I — M3) 


This formula took account of the sampling 
correlation between the correlation coeffi- 
cients or, stated differently, took account of 
the effect of using the same group. The cor- 
relation is not significant. 


Before 1936, the American Council Psy- 
chological Examination was the intelligence 
test used for masters’ students. When this 
test was used, the distributions of scores were 
negatively skewed. The distributions of scores 
on the CAVD more nearly approach a normal 
curve. In Colorado State College of Educa- 
tion, the CAVD Scale has proved to be a 
more practical scale for graduate students. 


_' Heilman, J. D. “The k and g Methods of Interpreting 
‘he Coefficient of Correlation.” Journal of Educational Psy- 
_ 28, 232-236 (March, 1937). 

* eters, Charles C. and Van Voorhis, Walter R. Statistical 
on edures and Their Mathematical Bases, p. 155, School of 
ucation, The Pennsylvania State College, State College, 


Pennsylvania, 193 


All students who elected the comprehensive 
examination plan and all who elected the four 
quarter plan with education as their major 
were required to take a battery of twelve ed- 
ucation tests and to pass any nine of the 
twelve. This battery of education tests con- 
sists of tests in the following subjects: tech- 
niques of curriculum making, elementary edu- 
cation, history of education, mental tests, 
personnel and guidance, philosophy of educa- 
tion, psychology of learning, research, school 
administration, secondary education, educa- 
tional tests, and statistics. The tests were 
made up by various faculty members of Colo- 
rado State College of Education and were 
revised by the Personnel Department of the 
college. There are two equivalent forms of 
each test. All the items of each test are 
multiple-choice of four choices. The number 
of items in the various tests ranges from forty 
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for research to fifty-seven for personnel and 
guidance. The passing mark on each test was 
determined from the group of ninety-one who 
first took the tests in the summer of 1936. 
The passing point was selected at approx- 
imately the 55th percentile. The percentile 
point for statistics and mental tests was some- 
what higher. 

\ comparison was made of scores on the 
CAVD Scale and scores on this battery of 
twelve education tests the first time they were 
taken. The quartile deviation (Q) of each 
education test was found, and the scores were 
given equal weights on the basis of the rel- 
then the average of the 
found, with nine as the 
minimum number of tests used. A product- 
moment coefficient of correlation (V 210) 
of .52 .034 was obtained between scaled 
scores on the CAVD Scale and this average 
of weighted scores on the education tests. 
indicates a substantial rela- 


ative sizes of the Q’s; 


weighted scores was 


rhis coefficient 
tion, 40 per cent better than pure chance. 
\ coefficient (.V 63) of —.377 .072 was 
found between scaled scores on the CAVD 
Scale and the total number of education tests 
which were taken before nine were passed. 
This shows a slight relationship between a 
small CAVD score and a large number of 
education tests taken in order to pass nine. 

A study was made of the relation between 
scores on the CAVD Scale and the choice of 
a program of graduate work leading to the 
master’s degree: (a) the thesis plan and 
(b) the four quarter and examination plans. 
Distributions of scaled scores on the CAVD 
tests were made for these two groups. The 
mean of the first group (V 323) was 
406.18 and the mean of the second group 
(NV 431) was 407.07. The difference be- 
tween these means was .89 and the probable 
error of the difference of the means was .49. 
There are 89 chances in 100 that the obtained 
difference is significant; this difference is in 
favor of the group selecting the second plan, 
but it is not a true difference. The standard 
deviation of the group which selected the 
first plan was 9.54 + .25 and of the second 
group 10.15 + .23. The difference between 


these two standard deviations was .61, and 
the probable error of the difference between 
these two uncorrelated o’s was .35. The 
ability as measured by the CAVD Scale is 
somewhat higher for the group electing the 
four quarter plan, but the obtained difference 
is not significant; the range of ability covered 
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by this group is slightly wider and again the 
difference is rot significant. 
4. CONCLUSIONS 

The IL.E.R. Intelligence Scale CAyp 
Levels M, N, O, P and Q is a discriminatin, 
instrument for determining the range of bj. 
ities of graduate students on the master: 
level. 

The correlation between the CAVD Sp. 
and teachers’ marks on the graduate level 
relatively low. The CAVD Scale is sufficient! 
discriminating, however, to make it virtual); 
certain that those students whose scores ar 
above the 75th percentile on the CAVD wi 
make higher marks on the average than thos 
students whose scores on the CAVD lie below 
the 25th percentile. 

There is a marked correlation between th 
CAVD Scale and the American Council 
Education Psychological Examination. Th 
correlation between the American Counci 
Psychological Examination and 
marks on the master’s level is slightly higher 
than that between the CAVD Scale and teach- 
ers’ marks, but neither is very high and the 
difference is not significant. It is more prac- 
tical to use the CAVD Scale for graduate stu- 
dents than the American Council Psycholog- 
ical Examination. When the American Coun- 
cil Psychological Examination was used for 
master’s students, the scores were negatively 
skewed. The scores on the CAVD more nearly 
approach a normal distribution. Evidently 
marks given to masters’ students in the sum- 
mer are based in the main on factors other 
than native ability as measured by intelli- 
gence tests. 

Although they evidently measure different 
abilities, substantial relationships were found 
between the CAVD Scale and the Coopera- 
tive English Usage Test, and between the 
CAVD Scale and the Cooperative Spelling 
Test. 

There is a marked relationship between the 
CAVD Scale and the battery of education 
tests. A slight negative correlation was found 
between scores on the CAVD Scale and the 
number of education tests that were taken 
before nine were passed. 

The scores on the CAVD of students who 
elect the four quarter plan for a masters de- 
gree are not significantly higher than the 
scores of those who elect the thesis plan. The 
first group shows a wider range of abilities 
than the second group. 
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PART I 
DEVELOPMENT OF A REGRESSION 
EQUATION* 
ng the past two decades a large num- 
of investigators have concerned themselves 
problem of the relationship between 
the students’ marks in college and _ their 
wks in high school, and the scores on vari- 
examinations which were administered 
r to their entrance into college. The main 
ose of these investigations has been to 
find criteria for accurate prediction of the 
student's achievement in college. The studies 
have shown definitely that it is impossible to 
foretell the scholastic achievement of all the 
students in a given group, and that many 
of success or failure can not be discov- 
ntil the student has tried to do college 
work. However, it would be advantageous to 
th the student and the college to determine 
s accurately as possible beforehand the stu- 
dent’s prospects for success or failure in 
college work. 
hese studies have shown a tendency to- 
ward basing the predictions of the student’s 
future status in college upon objective rather 
than subjective evidence. In some instances 
the zero-order coefficients of correlation were 
used to measure the relationship between two 
variables, in other instances the more exact 
procedures of partial and multiple correlation, 
path coefficients, and coefficients of determi- 
nation were used to determine the relationship 
between a dependent variable and a group of 
indepe at variables. Some _ investigators 
studied the procedure for one year; others 
studied the procedure for several consecutive 
years, and were able to determine the most 
valid predictors, the best single predictor, 
and the best group of predictors for their 
respective colleges. 


THE INVESTIGATION 


_ The purpose of this study is to develop a 
ive-variable regression equation which will 


Py A field study which was submitted in partial fulfillment 
if the requirement for the degree of of Philosophy in 
orado State College of Education, Greeley, Colorado, 1938. 


predict with as little error as possible the 
status of college freshmen at the time of their 
entrance into college. The equation is based 
on data obtained on 97 freshmen students at 
the La Salle—-Peru-Oglesby Junior College, 
La Salle, Hlinois, 1937-1938. The data in- 
clude the average college mark for the entire 
freshman year as the dependent variable, or 
criterion; and the American Council Psycho- 
logical Examination score, the Iowa English 
Aptitude Examination score, the Iowa High 
School Content Examination score, and the 
average high school mark for the four years 
as the independent variables. 

The zero-order coefficients of correlation, 
their probable errors, and the mean and sigma 
of each of the variables are shown in Table I. 
Leinbaum! found a coefficient of correlation 
of .52 between average college marks and 
average high school marks. Scott® reported a 
coefficient of .64 between average college 
marks and the American Council Psycholog- 


TABLE I 


ZERO-ORDER COEFFICIENTS OF CORRELATION BE- 
TWEEN VARIABLES USED IN THE PREDICTION 
OF THE AVERAGE COLLEGE MARKS OF 97 
FRESHMEN STUDENTS AT THE LA SALLE-— 
PERU—OGLESBY JUNIOR COLLEGE, LA SALLE, 
ILLINOIS, 1937-1938 


Variables: X,—the average college marks for 

the entire freshman year; 

X.—the American Council Psycho- 
logical Examination score; 

X;—the Iowa English Aptitude 
Examination score; 

X,—the Iowa High School Content 
Examination score; and 

X.—the average high school mark 
for the four years. 


X; Xs Xs 
57 
05 
56 
48 
.05 
52. 56 
05 05 
42.70 135.82 
7.00 32.86 


- 08 .05 
3.39 179.51 
Sigma __ 0.73 47.57 


393 








304 


ical Examination. Hovde* used percentile 
rank on the American Council Psychological 
Examination instead of raw score, and re- 
ported a coefficient of correlation of .55 be- 
tween the American Council Psychological 
Examination and average high school marks. 

In this study, the regression equation in 
score form is: X, == —.0002X, + .0163X, 
+- .0o59X, + .5217X, + .0378, (1). This 
means that the student’s predicted mark 
equals —.0002 times his score on the Amer- 
ican Council Psychological Examination plus 
.0163 times his score on the Iowa English 
Aptitude Examination plus .oo59 times his 
score on the lowa High School Content Exam- 
ination plus .5217 times his average high 
school mark plus the constant, .0378. 

By using this regression equation, the 
average college marks of the 97 students are 
predicted. There are three instances in which 
there are no differences between the predicted 
marks and the actual marks. These are in- 
stances of perfect prediction. The largest 
difference between the predictec and actual 
marks is 1.12 of a letter mark. The median 
of the differences is .26; the probable error 
of estimate, .30; and the standard error of 
estimate, .44 of a letter mark. All of the pre- 
dictions fall within three standard errors of 
estimate, 97 per cent within two, and 71 per 
cent within one; and 58 per cent within one 
probable error of estimate. 

The standard error of estimate of .44 means 
that two-thirds of the marks should be pre- 
dicted within .44 of a letter mark. For prac- 
tical certainty, the range of possible varia- 
tions must be increased to three sigmas, or 
1.32 of a letter mark. Therefore, if a student 
receives the predicted mark of “C”, the mid- 
point of the “C’’-range, the chances are 74 in 
100 that he will receive a mark of “C”, and 
it is practically certain that he will receive 
a mark of not more than “B”, nor less than 
es 

The coefficient of correlation between the 
predicted marks and the actual marks is .77, 
a significant relationship. Leuenberger* re- 
ported a relationship of .70. 

The coefficient of multiple correlation is 
computed from the regression coefficients and 
the zero-order coefficients of correlation. The 
coefficient of multiple correlation is .79*, and 
gives predictions 57 per cent better than pre- 


_* This is slightly higher than the .77 reported in the pre- 
vious paragraph on account of the grouping error affecting 


the predicted grades in the latter. 
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dictions by chance. in order to determine the 
total single and joint influences of the {o»- 
independent variables upon the criterion, th 
coefficients of determination are computed 
These coefficients are given in Table II. Th. 
total of the measured influences js 6243, oF 
62.43 per cent. In a normal distribution, th 
coefficient of multiple correlation squared 
equals the coefficient of determination. Ther 
is an approximate equality in this situation 
Heilman® reported a coefficient of multip\: 
correlation of .7985 and a coefficient of deter. 
mination of 63.73 per cent. 


CONCLUSIONS 


This study, although it contributes nothing 
new in the field of prediction, substantiate: 
other studies that have been made, and als 
demonstrates the use of correlations and the 
regression equation in prediction. 

The regression equation developed in this 
study is reasonably accurate, since the pre- 
dicted marks compare so favorably with th 
actual marks. This is tested by the appro: 
imate equality between the median of the 
differences and the probable error of estimat: 


TABLE II 


COEFFICIENTS OF DETERMINATION BASED ON 
DATA OBTAINED ON 97 FRESHMEN STUDENTS 
AT THE LA SALLE—PERU-—OGLESBY JUNIOR 
COLLEGE, LA SALLE, ILLINOIS, 1937-1938 


Variables: X:—the average college mark for 
the entire freshman year; 
X,—the American Council Psycho- 
logical Examination score; 
X:—the Iowa English Aptitude 
Examination score; 
X,—the Iowa High Schoo! Content 
Examination score; and 
X,—the average high school mark 
for the four years. 


Determinates Coefficients 

1 2 3 
dis ao er 0001 
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i. 2s MowBeu¥a ...-........- - 0020 
di. ES ee ee -.U049 
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The equality is not exact since the distribu- 


‘ion is not a normal distribution. Further 
‘ests are found in the significant relationship 
between the predicted and actual marks, in 
‘he number of predictions falling within one 
»robable error and one standard error of esti- 
‘sate, and in the equality of the means of the 
‘wo distributions. The distribution of pre- 
dicted marks has a smaller range and a 
smaller sigma. 

In this particular group of independent 
variables, the best predictor is the average 
high school mark. This variable makes the 
largest single and joint contributions, as is 
shown by the coefficients of determination. 
The poorest predictor is the American Coun- 
cil Psychological Examination, which makes 
the smallest single and joint contributions. 
The latter are all negative. The portion of 
the total influences on the criterion not meas- 
ured by the four independent variables is 
27.57 per cent. The addition of other tests, 
especially those which measure qualities and 
capacities not measured by the present tests, 
may reduce the percentage of influences un- 
accounted for in this study. 

Educational guidance presupposes some 
means of predicting college marks in order 
that the student may be directed into the 
type of work that best suits his capacities and 
needs. This regression equation can be used 
to predict the average college mark of 68 per 
cent of the students included in this study 
within the standard error of estimate of .44 
of a letter mark. 

This regression equation is significantly 
valid in predicting the average college marks 
of the students on whom the data were ob- 
tained. The real test of its validity will be 
made through using it in predicting for other 
similar groups of students from the same col- 
lege. In the subsequent investigation, the 
validity of the regression equation is verified 
by using it in predicting the average college 
marks of the freshmen students during the 
year following the development of the regres- 
sion equation. 


PART II 


THE VALIDITY OF THE REGRESSION 
EQUATION* 


The typical investigator chooses a group of 
variables and obtains their measures. He 


Pe field study which was submitted in partial fulfillment 
the requirement for the degree of Doctor of Philosophy in 
Colorado State College of Education, Greeley, Colorado, 1939. 
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uses these measures in an appropriate way to 
form a regression equation with which he 
predicts the probable status of all or a rep- 
resentative group of students on whom the 
data were obtained. If the regression equa- 
tion predicts well for the group, he recom- 
mends its use for further predictions in the 
same college, and considers his task com- 
pleted. 

The test of the regression equation has not 
been made at this stage. The investigator 
should make further predictions by obtaining 
like measures on a second similar group of 
students, and by using these measures with 
their respective weights from the first equa- 
tion. The coefficient of correlation between 
the predicted marks and the marks actually 
received by the students of the second group 
measures the validity of the regression equa- 
tion as a predictive instrument for the stu- 
dents at that particular college. 


THE INVESTIGATION 


This study is concerned with obtaining a 
measure of the validity of each of two five- 
variable regression equations used in predict- 
ing the probable average college marks of 
freshmen students during the year following 
the development of the regression equation, 
and with measuring the effect on the criterion 
of adding another variable to the variables 
upon which one of the regression equations is 
based. 

The subjects of the study, chosen by 
chance selection, are two groups of 100 stu- 
dents each. One group attended the La Salle— 
Peru—Oglesby Junior College, La Salle, Illi- 
nois, during the year 1938-1939; and the 
other attended the Colorado State Teachers 
College, Greeley, Colorado, during the year 
1932-1933. 

One regression equation was developed 
from data on ninety-seven freshmen students 
at the La Salle-Peru-Oglesby Junior College 
in 1937-1938. The equation (Part I) in score 
form is: 

X, = —.0002X, + .0163X, + .0059X, + 
.5217X, + .0378, (1) in which the 
variables are: 

X,—the average college mark for the 
freshman year; 

X,—the American Council Psychological 
Examination score; 

X, — the Iowa English Aptitude Examina- 
tion score; 
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X,—the Iowa High School Content Ex- 
amination score; and 

X the average high school mark for the 
four years. 

> the Personal 
added variable. 


Data Scale was the 


The other equation was developed by 
Leuenberger* from data on 283 freshmen stu- 
dents at the Colorado State Teachers College 
in 1931-1932. The equation in score form is: 


i o428X .2877X, .3356X, + 
.6429N, + 99.0944, (2) in which the 
Val lables are: 

Ba the average college mark for the 


freshman year; 
NX, —the American Council Psychological 
Examination score; 


X the English Test score; 
X the Elementary Test score; and 
\Y, —the percentile rank of the student in 


his senior class. 


Like measures on a second group of 100 
freshmen students from each college were 
obtained and used with the weights of the 
regression equation already developed in 
order to make the predictions. The differences 
between the predicted and actual marks were 
determined, and comparisons were made in 
terms of the differences and in terms of the 
per cents of the differences falling within and 
without the probable and standard errors of 
estimate. 

For each group of students, the zero-order 
coefficients of correlation between the pairs 
of variables were computed from the new 
data. The zero-order coefficients were used in 
setting up a set of simultaneous equations, 
the solutions of which gave the regression co- 
efficients. The regression coefficients were 
used in determining the coefficient of multiple 
correlation and the coefficients of determina- 
tion. The shrinkage of the coefficient of 
multiple correlation was also computed. 


SUMMARY OF FINDINGS 


For the La Salle-Peru-Oglesby Junior Col- 
lege students, the differences between the pre- 
dicted and actual marks ranged from .or1 to 
1.25 points, and the median of the differences 
was .33 of a point. Forty-nine per cent of the 
differences fell within one probable error of 
estimate, and 66 per cent fell within one 
standard error of estimate. The coefficient of 
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correlation between the predicted and ac 
marks was .78. In the previous study o;. 
similar group of students, the range was ; -. 
points, anc the median of the differences ys: 
.26 of a point. The per cents of the dif. 
ences falling within one probable and or. 
standard error of estimate was 58 per cen 
and 71 per cent respectively. The coefiicien 
of correlation between the predicted 
actual marks was .77. There were variation: 
between the respective measures of the ty 
groups: The range and the median of ty 
differences; and the per cents falling withip 
one probable error and one standard error 
estimate; but the variations were not dispro. 
portionately large. The regression equation 
developed from the data on the first grou; 
students maintained its validity in predicting 
the average marks of the second group. 

The coefficient of multiple correlatio; 
using the six variables, was .81: using th 
first five variables, it was .79. The presen 
of the added variable increased the coefiicient 
of multiple correlation. For the preceding 
year’s group, that coefficient was .79. There 
was a small estimated shrinkage of the 
efficient of multiple correlation, but no actu 
shrinkage. 

The total influences on the criterion as 
measured by the five independent variable 
was 67.26 per cent; as measured by the four 
independent variables it was 62.43 per cent. 
Singly and jointly, the Personal Data Scale 
measured 4.83 per cent of the total influences 
It measured singly .54 per cent of the total 
influences on the criterion, which was more 
than was measured by either the American 
Council Psychological Examination, or the 
Iowa English Aptitude Examination. 

For the Colorado State Teachers College 
students, the differences between the pre- 
dicted and actual marks ranged from .0o to 
1.38 points, and the median of the differences 
was .31 of a point. Forty-seven per cent 0! 
the differences fell within one probable error 
of estimate, and 65 per cent fell within one 
standard error of estimate. The coefficient 0! 
correlation between the predicted and actual 
marks was .70. In the original study 0! 4 
similar group, the range was .90 of a poitlt, 
and the median of the differences was .25 0 
a point. Fifty-one per cent of the differences 
fell within one probable error of estimate, n° 
66 per cent fell within one standard error 0! 
estimate. The coefficient of correlation be 
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-yeen the predicted and actual marks was .7o. 
n th ough there were increases and de- 
ogee in the range and the median of the 
erences, and in the per cents falling within 
ne probable error and one standard error of 
vate, the differences were not significant. 
regression equation was as valid in pre- 

ting for the second similar group of stu- 
ients as it was for the group for which it 
was deve loped. 
lh ficient of mul itiple correlation was 
“8 for the preceding year’s group it was .73. 
lhere was a small estimated shrinkage and 
large actual shrinkage of the coefficient of 
Jtiple correlation. The total influences on 
riterion as measured by the independent 
riables was 63.15 per cent, and increase of 
per cent over the total for the previous 


I. 
wa) 


CONCLUSIONS 

[his study reports a measure of the valid- 
ty of each of two regression equations used 
~ predicting the average college marks of a 

ind similar group of students. Each equa- 
ion was significantly valid for its own second 
group. 

[he best predictor in this combination of 
variables for the La Salle-Peru—Oglesby 
lunior College students was the average high 

ol mark; for the Colorado State Teach- 

s College students, the Elementary Test. 

American Council Psychological Exam- 

uation was the poorest predictor for both 
groups of students. 

lhe Personal Data Scale measured some of 
the total influences on the criterion, and 
measured slightly more than either the Amer- 
can Council Psychological Examination or 
the lowa English Aptitude Examination. 
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These regression equations should be an 
aid to those who advise the freshmen stu- 
dents who enter these two colleges, since they 
predict the average college mark of approx- 
imately 68 per cent of the students within .44 
and .40 of a letter mark. 
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EVALUATION OR GUIDANCE? 


The Report of the Eighth Annual National College Sophomore 
Testing Program April 17 to May 5, 1939 


Epwarp E. CuRETON 
Alabama Polytechnic Institute* 


I. INTRODUCTION 


American education has become evaluation- 
conscious. Objective tests and other instru- 
ments that are not so objective have been 
used and misused to evaluate individuals, in- 
structors, departments, colleges, and even the 
educational systems of entire states. Some of 
this evaluation is significant and valuable. 
Much of it is harmless and also useless. 
Certain progressive philosophers, observing 
that the useless and pernicious cases outnum- 
ber the significant and valuable ones, have 
condemned the whole evaluation movement, 
and the testing and guidance movements along 
with it for good measure. 

Guidance and evaluation are both examples 
of judgment. Evaluation looks backward and 
judges the past, while guidance looks forward 
and judges the future. Since the future, 
fundamentally, is always more important 
than the past, it follows that guidance is in- 
trinsically a more important type of activity 
than evaluation. Both of these activities 
employ measurement as a tool, and both of 
them employ other tools also. One still hears 
occasionally the argument that since compar- 
able objective tests do not constitute a com- 
plete kit for either purpose, they should be 
abandoned. Fortunately, it is hardly neces- 
sary any longer to reply to this argument. 

Evaluation, contrary to popular belief, 
makes much more exacting demands upon the 
science of measurement than does guidance. 
If a test is to be used in the appraisal of the 
effects of instruction upon an individual, a 
class, or some larger group, it must possess 
demonstrable validity for its purpose. This 
means that it must measure with some fidel- 
ity the really important outcomes desired 
from the instruction. These outcomes may be 
in the nature of skills, items of knowledge, 
habits, attitudes, or even character and per- 
sonality traits. Existing tests are much more 
successful in measuring the first three of these 

* Formerly Research Advisor, Cooperative Test Service. 


outcomes than the last two. Philosophers 
agree unanimously, on the other hand, that 
the last two are more significant than the 
others. If the work of students and teachers 
is evaluated primarily on the basis of obje- 
tive tests, therefore, there is an inevitable 
temptation to neglect the more fundamental 
objectives in favor of the more measurable 
ones. 


The obvious way out of this dilemma is to 
appraise a// of the more important desired 
outcomes, using the most accurate methods 
available in each case. In arriving at general 
judgments, it is then necessary to weight each 
appraisal by its intrinsic importance pri- 
marily, and only secondarily by the reliability 
of the measurement or estimate on which it 
is based. The fact that it is difficult to do 
this has occasionally been used as an argu- 
ment to support proposals that objective tests 
be abandoned as instruments of evaluation 
If the difficulty is real, and apparently it : 
real in many cases, it would seem much more 
logical to be cautious about the whole evalu- 
ation process. Judgments about the real 
growth of students toward worthy ends, 
about the effects of specific programs of in- 
struction upon such growth, about the relative 
abilities of different instructors to stimulate 
the process, and about the general excellence 
of a school or school system as an environ- 
ment for nurturing it, should be offered only 
with the greatest caution. Evaluation is 4 
very difficult art, and the tools at present 
available are not only crude on the whole, 
but of highly uneven crudity as well. 

Guidance, on the other hand, imposes 
appreciably simpler requirements upon the 
technique of measurement than does evalua 
tion. To possess value as an instrument 0! 
guidance, a test needs only to measure some 
function that is important, and to give a more 
accurate appraisal than is provided by other 
measures of this or of related functions. Tests 
can easily be dangerous as instruments © 
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»yaluation, but as instruments of guidance, 
‘hey are almost always useful unless grossly 
misinterpreted. Suppose that a group of stu- 
septs are measured by a battery such as the 
~snerative General Culture Test, contain- 
ng ‘ymong others survey tests in the social 
dies and in the natural sciences. Suppose, 

+her, that these two tests measure only 
facts and the more or less 


wledge ol 


vious interpretations of these facts, but 
‘hat they are both provided with accurate 
and comparable norms. The social science 
nartments of the college might be engaged 


‘marily in efforts to develop in their stu- 
jents the attitudes which are the logical con- 
mitants of a democratic philosophy, and 
.e natural science departments might at the 
ame time be bending their energies toward 
vying their students a general appreciation 
i the effects of science upon modern indus- 
| civiization and of the necessity for the 
nsion of scientific modes of thought into 
consideration of social questions. If, now, 
: were proposed to “evaluate” the two de- 
artments by comparing the average score in 
each with the corresponding norm, great 
rm might easily be done. If the “evalua- 
tions” were made the basis for any effective 
ministrative action, both groups might well 
be induced thereby to abandon their progres- 
sive policies and go back to the compart- 
entalized teaching of facts, principles, and 
vious interpretations within their respective 
eas 
Now consider the individual student. He 
may be shown nis scores on these two tests 
n terms of some comparable units, and it 
may be explained to him that the tests 
measure only factual and elementary inter- 
retative knowledge in their respective areas. 
(his will not change any of the desirable 
ttitudes and appreciations that may have 
een built up in him as a result of his work 
the various departments. The added 
<nowledge concerning his background in the 
two general areas will be a clear gain to him, 
making his future educational decisions and 
ossibly even his basic vocational choices 
‘hat much more intelligent. 


The reports from participating colleges 
indicate that up to the present time the re- 
sults of the national sophomore testing pro- 
grams have been used primarily for purposes 
of evaluation. Many colleges have been con- 
‘ent with this, and have continued to partici- 
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pate in the programs year after year. Many 
others, after participating for a year or two, 
have found as much harm as good resulting, 
or have discovered that after giving the tests 
they still possess too inadequate information 
for really effective evaluation, and have 
therefore withdrawn from the programs. 
Some few, possessing adequate personnel and 
guidance staffs themselves, have used the 
tests regularly for guidance purposes, sup- 
plying the necessary interpretative materials 
themselves. 


A definite effort has been made by the 
Cooperative Test Service this year to increase 
and emphasize the guidance values of vie 
tests used in this program. This effort has re- 
sulted in three innovations. First, the Co- 
operative Contemporary Affairs Test was 
modified to yield six part-scores, on the 
assumption that the pattern of these scores, 
after reduction to comparable units, would 
yield an estimate of the corresponding pat- 
tern of functioning interests of the student. 
A knowledge of these functioning interests, it 
was felt, should prove a valuable supplement 
to the corresponding measures of academic 
achievement in enabling the student to ap- 
praise his total background. Second, prelim- 
inary norms were derived at the earliest pos- 
sible date, and the General Culture and Con- 
temporary Affairs Tests were provided with 
scores approximately equivalent to the Scaled 
Scores of the Cooperative English and Liter- 
ary Comprehension Tests. Third, special pro- 
file charts were drawn up, incorporating these 
Scaled Scores and Scaled Score Equivalents, 
and an effort was made to supply each co- 
operating college with copies of these charts 
before the end of the spring semester. Each 
college was sent without charge as many 
copies of the chart as the number of students 
tested. It was hoped that these charts would 
be filled out and interpreted by the students 
themselves. 


In the second section of this article, we 
present the usual normative and statistical 
data required for purposes of evaluation, to- 
gether with some evaluations of the tests. In 
the third section, an effort is made to discuss 
in greater detail the techniques of using com- 
prehensive comparable tests for guidance 
purposes, with particular reference to the 
needs of colleges which do not possess special 
personnel and guidance officers. 








Il. EVALUATION 


A total of 28,461 students in 159 colleges 
and universities were measured with one or 
more tests in connection with the 1939 
National Sophomore Testing Program. Of 
these, 118 institutions returned their results 
in time to be used in the tabulations upon 
which this report is based. Since the partici- 
pating colleges used various combinations of 
tests, the tables following are all based on 
different numbers of cases. It is believed, 
however, that the results are approximately 
comparable except for the greater accuracy 
of those based on larger numbers of cases. 


Bastc NORMATIVE DATA 


The tests used in the 1939 Program, as in 
previous years, were divided into two sets. 
Those in the first, which were considered the 
most important, were recommended to all 
participants as a minimum program. They 
consisted of the English, Literary Compre- 
hension, General Culture, and Contemporary 
Affairs Tests. A variety of tests was recom- 
mended in the second set, as supplementary 
examinations to be used as local conditions 
seemed to warrant. This report is concerned 
only with the tests of the minimum recom- 
mended program, since no particular one of 
the other recommended tests was used in a 
sufficient number of colleges to warrant the 
preparation of special norms, or the under- 
taking of interpretative studies. 

Table I presents the final percentile norms 
for the tests of the minimum recommended 
program. These norms are based on the re- 
sults returned from coileges in which the Gen- 
eral Culture and Contemporary Affairs Tests 
were administered by the usual process of 
having the students mark their answers in the 
test booklets. Since the English and Literary 
Comprehension Tests are provided with 
Scaled Scores, the tables on the keys and an- 
swer sheets for converting raw scores into 
Scaled Scores provide for any differences in 
the modes of administration of these two 
examinations. 


The growing use of the International Test 
Scoring Machine is reflected in the increasing 
number of colleges using separate answer 
sheets with the General Culture and Contem- 
porary Affairs Tests. While a few colleges 
used separate answer sheets and scored these 
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by hand, it is highly probable that jy 

of the institutions where separat, meni 
sheets were used with these tests, the scor a 
was done by machine. In 35 different ¢,). 
leges, a total of 11,789 students used answer 
sheets with either the General Culture Tox 
or the Contemporary Affairs Test 
Twenty-one of these institutions returned 
sults in time for tabulation. Table I[. w 
is based on these returns, gives the bas) nee. 
centiles for the two tests. It has been estab. 
lished in a number of cases that the yse 0 
separate answer sheets increases the difficy): 
of any test with which they are employed, s 
that separate norms are necessary. The dif. 
ferences are usually not very large, and the, 
vary appreciably from one test to another 
The fact that the scores at most percentiles 


in Table II are higher than the correspond- 
ing scores in Table I indicates, therefore, that 
the 21 colleges using separate answer sheets 
which returned results are appreciably above 


the general averages shown in Table | 
Equatings were made by two different metb- 
ods to determine the additional difficulty of 
the General Culture and Contemporary 
Affairs Tests when used with separate answer 
sheets. The results of this equating are 
presented in Table III. 


In a number of institutions, the Coopera- 
tive General Culture Test, besides being used 
in the sophomore class, is used in lower and 
higher classes as well, and rough measures o! 
educational growth are obtained by noting a 
student’s gains in score from year to year. 
In order to interpret these measures, it is 
necessary to know what score on one form 0! 
the test corresponds to a given score on ab- 
other form. Table IV gives the scores on 
Form P, used this year, which are equivalent 
to given scores on Form O, used last year. 


In a few institutions, the tests of the min- 
imum recommended program were given t 
all four classes. Even in these institutions, 
however, not all of the tests in the program 
were used. Table V gives condensed percen- 
tiles for all four classes in these colleges. 
Since the number of cases is rather small, 
these results will not be extremely accurate. 
A rough idea of the representativeness of the 
tables may be obtained by comparing per 
centiles for the sophomore class with the cor 
responding percentiles in Table !. 
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INDIVIDUAL AND INSTITUTIONAL and Contemporary Affairs Tests. On th 
DIrFERENCES whole, the spread of the college averages 

Ihe tremendous variability in the back- only a little less than half as great as that 
grounds, abilities, and interests of college of the individuals. The highest college in 
students is by now well known. The fact that General Culture is above the ninetieth per. 
there are very large differences between in- centile, and the lowest college in Contempo. 
stitutions, as well as between individuals rary Affairs is below the third percentile {»; 
within the same institution, is often not ap- all individual students. In spite of this grea: 
preciated so clearly. Chart I shows the vari- variability among college averages, however 
ability of all individuals compared with the _ it should be noted that within any particy!s; 
variability among college averages for the college the variation among individual sty. 
total scores on the English, General Culture, dents may still be expected to be more thy: 


CHART I 
COMPARISONS OF THE VARIABILITY OF INDIVIDUALS AND OF COLLEGE AVERAGES 


Explanation: The bars in each section of the graph show the variation among individuals 
and among college averages, respectively. The wide portion of each bar represents the range 
of scores of the middle half of the students, or the middle half of the college averages, on the 
designated test. The narrow parts extend to the 16th and 84th percentiles, and the lines at the 
ends extend down to the tenth and up to the 90th percentiles. In the case of the college aver- 
ages, the cross below each bar represents the lowest college average, and the one above rep- 
resents the highest college average. The short cross line near the middle of each bar indicates 
the median college average. 

While this chart is based entirely on percentiles, the scale has been altered to correspond 
roughly to a sigma scale, so that vertical distances are approximately comparable. 


ENGLISH GENERAL CULTURE CONTEMPORARY AFFAIRS 
Ind. Coll. Ind. Coll. Ind. Coll. 


Vari- Aver- Vari- Aver- Vari- Aver- 
ation ages ation ages ation ages 
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ss per cent as great as the variability of 
students in all institutions taken together. 
The fact that large institutional differences 
exist does not greatly lessen the problem of 
‘dividual differences within any given col- 
lege or university. 
The existence of such large differences 
between colleges has a number of interesting 
‘»plications. It is evident that these insti- 
~tions must possess different academic aims, 
since they serve students of such different 
ackgrounds and accomplishments. This is as 
: should be. Variation among institutions is 
sential if a wide variety of college students 
are to be served adequately. But the old fic- 
ion that a count of credit-hours or a tran- 
script of grades furnishes a comparable rec- 
rd of achievement or ability must be 
sbandoned. One of the most important prac- 
| uses for comparable comprehensive test 
scores, in fact, is in connection with transfer 
students. A grade-transcript indicates a stu- 


dent’s accomplishments in terms of the aca- 
demic standard of the institution he has been 
attending. This should be supplemented by a 
test-transcript, showing his accomplishments, 
in the areas measured by the tests, in terms 
of the academic standard of the institution 
he proposes to enter. We need more variation 
among the academic standards of colleges, 
not less. But this variation should be con- 
trolled, deliberate, and open. The ostrich- 
attitude that all “accredited” colleges have, 
or even should have, roughly similar academic 
standards ought to be abandoned once and 
for all. 


COMPARISONS OF SPECIAL GROUPS 


In order to make the following compari- 
sons, a restricted group was selected from 
among the students whose norms are given 
in Table I. This group was composed of 
4588 individuals; 1878 men and 2710 
women. It consisted of all students who had 


CuHaArtT II 
ACHIEVEMENT OF MEN AND WOMEN STUDENTS 


Explanation: This chart shows the means of 1878 men and 2710 women students (soph- 
res) on the indicated tests, graphed in terms of national percentiles as in Chart I. 


Test or Subtest 


General Culture 


Science 


Contemporary Affairs 
} 3 





4 
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taken the English, General Culture, and Con- 
temporary Affairs Tests, and who had filled 
out all of the information requested on the 
cover-pages of the last two. In order to ob- 
tain strict comparability in the results, it was 
limited to individuals who had not used sepa- 
rate answer sheets with either the General 
Culture or the Contemporary Affairs Test. 
The various comparisons which follow have 
been made by constructing profiles of the 
average scores on the subtests of the English, 
General Culture, and Contemporary Affairs 
examinations for each set of groups to be 
compared. 


Men and Women.—This first and basic 
comparison is shown on Chart II. On the 
English examination, the women are appre- 
ciably superior in usage and spelling and not 
appreciably inferior in vocabulary. On the 
General Culture Test, the men are superior 
in knowledge of social studies, science, and 
mathematics, and the women in knowledge of 
literature and fine arts. A similar situation 
is found for the Contemporary Affairs Test, 
where the men are superior in knowledge of 
political events, social and economic events, 
science and medicine, and amusements; while 
the women are superior in knowledge of re- 
cent happenings in the fields of literature and 
fine arts. These findings support the general 
observation that women are superior to men 
in the linguistic and artistic fields, while men 
are superior to women in the technical fields 
and in what might be termed general infor- 
mation. In the case of the fine arts test of 
the Contemporary Affairs battery, it will be 
noted that both the men’s and women’s aver- 
ages are above the soth percentile. This 
means that the 4588 individuals in the re- 
stricted group are not exactly representative 
of the entire 5701 used in compiling the 
norms. For all of the remaining comparisons 
in this set, the charts were drawn separately 
for men and for women. 


Major Fields of Study.—On the cover page 
of the General Culture Test, there appears 
the question, “What is your major field of 
study?” Ten possible answers to this question 
are listed, and a space is provided for writing 
in an eleventh. The item “miscellaneous” on 
the chart refers to items listed in this last 
category. Charts III and IV give the profiles 
of average scores of students majoring in these 
various fields. The most noteworthy feature 
of Chart III is the extreme variability of 
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these groups on the literature and mathe. 
matics tests in the General Culture battery 
The highest scores in literature, strange) 
enough, are found among the group majoring 
in classical and modera languages, rathe: 
than among those majoring in English. This 
may be due to the fact that the test measyre: 
knowledge of world literature, rather than ; 
English and American literature specifical)) 
The highest peak on the chart, natural 
enough, is the point representing the super. 
ority of mathematics majors on the mathe. 
matics test. Chart IV shows much less vari 
ability on the whole than does Chart II] 
This may indicate that women base the; 
choices of major fields upon less careful self- 
appraisals than do men, or it may merel 
show that women exhibit a more uniform 
scholastic motivation than do men. The out 
standing feature of Chart IV is again th 
mathematical superiority of mathematics 
majors. The superiority of the music majors 
on the fine arts sections of both the General 
Culture and Contemporary Affairs Tests is 
also noteworthy. The most consistently high 
scorers are the men classical students, and th: 
most consistently low scorers are the vy 
students of business administration. 


Professional Goal.—A second question on 
the cover page of the General Culture Test 
is “What is your professional goal?” Nine 
professional goals and one additional cate- 
gory, “Uncertain’’, are listed as possible an- 
swers. Charts V and VI give the profiles o! 
men and women students checking these 
various professional goals. Men students o! 
engineering show the expected superiority in 
science and mathematics and a concommitan! 
inferiority in English and social studies. 
Since there are only two women students 0! 
engineering in the entire group, no profile has 
been plotted for this subgroup. The most 
noteworthy fact brought out by these charts 
would seem to be the consistently high scores 
of students of journalism. In the case of men 
students, this superiority is reduced sull- 
ciently on the fine arts, science, and mathe- 
matics sections of the General Culture Test 
to put their total scores below those of the 
college teaching group on that test. The 
women students of journalism, on the other 
hand, are almost at the top of the fine arts 
section, as well as of the social studies and lit- 
erature sections, and they are above the aver- 
age even on the science and mathematics 
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cections. Their total scores on the General 
Culture Test, therefore, as well as on the Eng- 


sch and Contemporary Affairs tests, exceed 
‘hose of any other group. It is of particular 
nterest to note that the “uncertain” groups 
-re near the medians for both men and women. 


rhis would seem to call in question the state- 


~ent, sometimes made, that students who 
have a definite professional goal tend to do 
-onsistently better work than those who do not. 


Cultural Influences ——A third question on 
the cover page of the General Culture Test 
is. “Which of the following do you think has 

ntributed most to your general cultural 
evelopment while in college?” Eight possible 
answers are listed, and a space is provided 
for writing in additional ones. Charts VII 
ind VILL give the profiles of students having 
various opinions on this subject. On the 
men’s chart, one is struck immediately by 
the consistent superiority of those who believe 
that independent reading has been the great- 
est contributor. At the bottom we find, also 
with considerable consistency, those who be- 
ieve that the principal contributor was labo- 
ratory work or tutorial or small group confer- 
ences. On the women’s chart, on the contrary, 
the group who believe tutorial or small group 
conferences to have been the greatest con- 
tributor to their cultural development have 
on the whole the highest scores of any. Their 
scores are hardly superior, however, to those 
of the group who believe independent reading 
to have been the greatest contributor. At the 
bottom of the women’s chart we again find 
laboratory work. Probably the reason that a 
belief in the cultural value of tutorial or 
small-group conferences characterizes good 
women students is that the group contains 
certain women’s colleges with a highly selec- 
tive admissions policy which use the tutorial 
system. 


Preferred Periodical—On the cover page 
of the Contemporary Affairs Test there ap- 
pears the item, “Encircle the number corre- 
sponding to the periodical which you like 
best to read.” Ten periodicals are listed. 
Though no additional space was provided, a 
considerable number of students wrote in 
some other name, and these were all classified 
as “miscellaneous” in the preparation of 
Charts IX and X. No very clear trends seem 
apparent in these charts. Men readers of the 
Scientific American are superior in science 
and mathematics, as might be expected, and 
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men readers of the Saturday Review of Liter- 
ature seem to Le somewhat superior in gen- 
eral to the readers of other magazines. This 
finding is even more pronounced in the case 
of women readers, but oddly enough, the 
readers of Poetry appear fairly close to the 
bottom of the group of wemen. 

Preferred Activities—The cover page of 
the Contemporary Affairs Test contains also 
the request, ‘“Encircle the number correspond- 
ing to the type of activity in which you are 
most interested.” Five groups of activities are 
listed, corresponding to the five subtests of 
the Contemporary Affairs Test other than 
amusements. Charts XI and XII show the 
profiles of those who prefer these various 
types of activities, and an added category 
“miscellaneous” has been added to cover 
other preferred activities written in. The one 
obvious and noteworthy feature of these 
charts is the definite superiority of those ex- 
hibiting a preference for the activity, “Writ- 
ing for student publications: belonging to a 
literary club.” 


Fields of Interest—A third item on the 
cover page of the Contemporary Affairs Test 
is the request, “Encircle the number corre- 
sponding to the field in which you are most 
interested.” Five fields are listed, correspond- 
ing almost precisely to the five subtests other 
than amusements on the Contemporary 
Affairs examination, and a sixth category was 
added on Charts XIII and XIV to include 
other fields of interest written in. The con- 
sistently high scores of those professing in- 
terest in literature is again a noteworthy fea- 
ture. A comprehensive study of the validity 
of the Contemporary Affairs examination as 
a measure of functioning interests is now in 
progress. 

Summary.—Throughout these various com- 
parisons, the high scoring students have been 
those who seem to possess linguistic interests. 
Students who major in the classics, who in- 
tend to become journalists, who believe that 
independent reading has contributed most to 
their cultural development while in college, 
who prefer to read the Saturday Review of 
Literature, who like to write for student pub- 
lications and belong to literary clubs, and who 
profess a general interest in literature seem to 
make high scores oftener than do the others. 
The exceptions are usually their scores in sci- 
ence and mathematics. All this would seem to 
argue for the point that linguistic ability is 
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rather general, affecting knowledge of the 
social studies, the fine arts, and contempo- 
rary happenings as well as of English. It 
would also seem to suggest that the test bat- 
tery is heavily !oaded with measures of this 
trait. 


COMPARISONS OF GOooD AND Poor STUDENTS 


For the following comparisons, two new 
groups were obtained, consisting approxi- 
mately of the highest and lowest tenth of the 
students in each college concerned in the pre- 
vious comparisons. This procedure was used, 
instead of the alternative one of picking the 
highest and lowest ten per cent of individuals 
from the total group, in order that differences 
between one institution and another should 
not enter into the comparisons. The group 
hereafter designated as the “upper ten per 
cent” consisted of the 455 individuals, from 
the total group of 4588, having the highest 
total scores on the Cooperative General Cul- 
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ture Test in their respective colleges. 7; 
group to be called the “lower ten per cent 
consisted of the 455 constituting the ten »»- 
cent, approximately, in each college, ha 
the lowest total scores on this examinat)- 
For these two groups various comparis 
have been made in terms of the data re 
on the cover pages of the General Culture 4: 
Contemporary Affairs examinations, and ; 
Contemporary Affairs Test profiles of the 
groups have becn plotted also. 
Contributions to Cultural Development 
The numbers in the upper and lower ten ; 
cent checking the various contributors : 
cultural development are shown in Chart X\ 
The only iarge difference on this chart 
seem to be the one concerned with indepe; 
ent reading. Good students appear to rate 
independent reading much higher than ¢ 
poor students. The poor students rate 
discussion a little higher than do the ¢ 
students, and both groups rate independen' 


Cuart XV 


APPRAISALS OF CONTRIBUTORS TO CULTURE MADE BY GOOD AND PooR STUDENTS 


Explanation: This chart is based on the replies of students who ranked in either t 


highest or the lowest tenth of the distribution of students in each college on the Genera 
Culture Test. The numbers at the left indicate the number of times each factor was checked 


by students who felt that this factor had contributed most to their cultural development. 


Contributors to Cultural Development 


Informal Faculty 
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CuartT XVI 


PERIODICALS PREFERRED BY GOOD AND POOR STUDENTS 


»»ylanation: This chart is based on the replies of students who ranked in either the 


Lt} 


- or the lowest tenth of the distribution of students in each college on the General 


_ vase Test. The numbers at the left indicate the number of times each periodical was 


ecked by students preferring it. 


Preferred Periodical 
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reading, class discussion, and course lectures 
more highly than they rate laboratory work, 
tutorial or small group conferences, extra- 
urricular activities, informal discussions with 
students, or informal discussions with faculty. 

Preferred Periodical—-The numbers in the 
two groups who prefer to read various peri- 
dicals are shown in Chart XVI. No large 
‘ifferences appear here, though the preference 
{ good students for the Scientific American 
may be slightly significant. 

Preferred Activity—Chart XVII shows the 


ent types of activities. Appreciable differences 
ccur in these groups. The good students 
appear to prefer literary and scientific activ- 
ites appreciably more often than do the poor 
students; and the poor students appear to 
preter social and economic activities notice- 
ably more often than do the good students. 


Field of Interest —Chart XVIII shows the 
numbers in the two groups having different 
stated major fields of interest. Here again 
noticeable differences appear. Good students 
seem most often to be interested in literature 
or in science and medicine, while poor stu- 
dents are interested appreciably oftener in 
social and economic events. 

Contemporary Affairs Profile-—Chart XIX 
shows the profile of average scores on the 
Contemporary Affairs Test of students in the 
upper and lower ten per cent on the General 
Culture Test. From this chart, it is obvious 
that good students know more about Con- 
temporary Affairs than do poor students, and 
that the difference is relatively uniform from 
one field to another. The good students show 
an expected slight relative dip on amuse- 
ments, though they still surpass the poor stu- 
dents on this item by a substantial amount. 
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Cuart XVII 
ACTIVITIES PREFERRED BY GOOD AND PooR STUDENTS 


Explanation: This chart is based on the replies of students who ranked in either ; 
highest or the lowest tenth of the distribution of students in each college on the Genera 
Culture Test. The numbers at the left indicate the number of times each activity was py 


ferred by these students. 


Preferred Activity 
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Ill. GUIDANCE 


lhe guidance movement has been growing 
steadily in recent years. Its development has 
not been as rapid as has that of the evalua- 
tion movement, but on the other hand it has 
been somewhat more consistent. Opposition 
has been passive rather than active; a matter 
of inertia rather than of any lefinite objec- 
tions. It has been easy to misuse tests in eval- 
uation, but the respect for personality which 


is an integral aspect of the American ce 
cratic philosophy has been successful in re- 
straining guidance officers and teachers tro! 
placing unwarranted confidence in test scores 
in the actual counseling situation. 

The major obstacle in the way of the ef 
tive development of the guidance movemen! 
has been, if anything, a too great skeptics! 
regarding the value of objective tests for ts 
purpose. This skepticism has been effective) 
reinforced by the consideration that pro'e 
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CHART XVIII 
INDICATED FIELDS OF INTEREST OF GOOD AND PooR STUDENTS 


Ez) 


{ 


checked by these students. 


lanation: This chart is based on the replies of students who ranked in either the 
-nest or the lowest tenth of the distribution of students in each college on the General 
siture Test. The numbers at the left indicate the number of times each field of interest was 
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sional personnel services are usually quite ex- 
pensive. While there is no doubt of the value 
t such services when available, it is perhaps 
not recognized generally enough that a great 
deal of effective guidance can be given by 
regular instructors and deans, and that with 
some slight assistance, college students can 
make intelligent decisions themselves on the 
vasis of knowledge of their test scores. 

The primary requisites for student self- 
guidance are a battery of tests covering a 


sufficient variety of fields, a system for ex- 
pressing the scores on these tests in compar- 
able units, and some means for helping the 
examinee to picture the pattern of his own 
scores. In planning the 1939 National Sopho- 
more Testing Program, definite efforts were 
made to meet all three of these requirements. 

The first step consisted in the broadening 
of the basic battery of tests. This was done 
in several ways. The General Culture Test 
had been revised the year before to include 
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CHART XIX 
COMPARISON OF SCORES ON THE CONTEMPORARY AFFAIRS TEST OF GOOD AND Poor Srvppy- 


Explanation: This chart shows the profiles of average scores on the subtests of the c.. 
temporary Affairs Test of men and women who ranked in either the highest or the lowes: ;,. 
per cent of the distribution of students in each college on the General Culture Test 7. 
averages are plotted in terms of national percentiles as in Chart I. : 
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sections on science and mathematics, as well edge of current happenings—one in the iielcs 
as on social studies, foreign literature, and of political, social, and economic events, a0¢ 
fine arts. This year it was further revised by _ the other in the areas of literature, music, 20 
making the literature test cover more or less art. In the 1939 edition, six different su> 
the whole field of world literature. The tests were included. The first two covere: 
greatest change in the test battery however, government and politics, and social and ec 
came in the revision of the Contemporary nomic events. The third dealt with curres' 
Affairs examination. This test had formerly happenings in the fields of science and mec 
provided only two general measures of knowl- icine. The fourth and fifth covered contemp 
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CHART XX 


Explanation: This is a copy of the Individual Profile Chart, filled out with actual data 
from ‘the scores of a student in one of the colleges participating in the Sophomore Program. 


<pECLLL PROFILE CHART FOR COOPERATIVE TEST SCORES: 1939 NATIONAL COLLEGE SOPHOMORE TESTING PROGRAM 
i 


ORM L For Students Who Recorded Responses to General Culture and Contemporary Affairs Tests in the Booklets 


ar 





CONTEMPORARY AFFAIRG 
terest Pattern 








ONS. It is suggested thet each student fill in his own test results VALUE OF THE CHART. This Chart enables the student to appraise his 
t. For the English and Literary Comprehension Tests, he should own performances in various ways: 
es eround the dots in the appropriate columns, at the levels of his 1. He can compare his score on any test with the Scaled Score 50. 
res in these subjects. The Scaled Score values are shown in the which is the score that the “average American” (1.O. 100) would be expected 
t the left of to make upon graduating from an average high school. 
left of the Chart. For the General Culture and Contemporary 
t ts cals aesamane * ircle th cat 2. He can compere his score on any test with the scores of sophomores 
id ee pe See ee Seenaeree Cee See ne in his own and in other institutions by means of the local and national 
ther tests which yield Scaled Scores may be entered in the percentiles 


‘one! spaces provided, and the correct dots encircled by reference to 3 


4¢ 


He can study his own relative achievements and functioning interests 
@ volves in the column at the right of the Chart. in different fields by direct visual inspection of his profile, since all tests ore 
equated to the Scaled Score units. This is perhaps the most important use 
of the Chart, and in fact the most important purpose of the National 
os Sophomore Testing Program, since it gives the student direct aid in ep- 
of the Chart. to facilitate further comparisons. praising his own particular strengths and woaknesses 


rcentiles may be entered in the spaces at the extreme right. 
reont 


ss. if these have been computed, in the spaces at the 
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rary literature and fine arts; and a sixth sec- 
tion was added dealing with radio, moving 
pictures, and sports under the general head- 
ing of amusements. All of the items making 
up this test were on events of the past year 
only, so that the scores on the various sec- 
tions should indicate the student’s function- 
ing interests in the corresponding fields, 
rather than his specific instruction in college. 
While the total score on a test such as this 
seems to show primarily the general alertness 
of the individual, it was felt that the pattern 
of scores on the six subtests might reflect the 
corresponding interest pattern as this reveals 
itself in information actually learned and re- 
tained. If the Contemporary Affairs Test pro- 
file docs reflect the pattern of these function- 
ing interests, the use of this test should add 
considerably to the value of a guidance bat- 
tery based on tests the rest of which are 
measures of school achievement primarily. 
Given an adequate battery of tests, the 
utilization of their scores by the students 
who take them demands some simple and 
comparable system for expressing these 


scores. A number of the Cooperative tests 
have been provided with a special system of 


Scaled Scores. Among these are the English 
and Literary Comprehension examinations. 
The Scaled Scores are directly comparable 
from one form of a test to another and from 
one examination to another. While the Gen- 
eral Culture and Contemporary Affairs Tests 
have not been scaled directly as yet, approxi- 
mate Scaled Score Equivalents were derived 
as early as possible on the basis of about the 
first 5000 sets of scores sent in. These Scaled 
Score Equivalents were obtaired by consider- 
ing the percentiles on the various subtests of 
the General Culture and Contemporary 
\ffairs examinations equivalent to the cor- 
responding total Scaled Score percentiles on 
the English Test. 

With comparable scores provided for all 
the tests of the minimum recommended pro- 
gram, it was still necessary to devise some 
method of making them available to the indi- 
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vidual students in an easily interpreted fo. 
A special profile chart was prepared for + 
purpose, and copies of this chart wer, 
tributed to all the participating colleges 
numbers sufficient so that one could be give 
to every individual student. Chart XX « », 
produced directly from one of thes 
charts, filled out with actual data from thy 
scores of a student in one of the participa: ~ 
colleges. The range of scores of this stydor 
is tremendous, varying from a spelling ; 
below the fifth percentile (and just 
average for the eighth grade) to a sci 
score above the goth percentile for the to: 
sophomore group. Since this student's math. 
ematics score is almost as high as his scien 
score, and since his science and medicirs 
score on the Contemporary Affairs Test 
also his highest in this set, it would appes 
that his professional goal of engineering ha 
been wisely chosen. This finding is corrob- 
rated further when we note that his major 
field of study as an undergraduate is n 
matics, he believes course lectures t 
been the major contributor to his cultura 
development, he prefers the Scientific Am: 
ican to other magazines, and he lists scies- 
tific interests and activities as appealing ' 
him most. 

It is to be hoped that eventually colleges 
will come to realize that the greatest value o' 
any testing project—as of any other phase 
the educational program—is its value to th 
individual student. The major purpose 
higher education in a democratic society is ' 
assist as many members of the oncoming 
generation as possess the requisite background 
in the development of their potentialities for 
effective personal living and for useful service 
to society. The purpose of guidance is to hel 
each student to appraise his abilities, inter- 
ests, and background; to plan his further ecu- 
cation intelligently in the light of this ap 
praisal; and to select and prepare himself ‘0: 
a vocation which will be suited to his poten- 
tialities, his limitations, and his interests 
Guidance and instruction are twin aspects 0! 
any adequate educational program. 








CORRECTIONS TO CORRELATION COEFFICIENTS 
ON ACCOUNT OF HOMOGENEITY IN ONE VARIABLE 


TEOBALDO CASANOVA 
Department of Education of Puerto Rico 


iiten it is necessary to use a sample 
‘ncludes several grades or several age 
rder to find the correlation between 
ther traits, say XY and ¥. This may 
»ypen because the sample in one grade or 
age class is too small to determine the cor- 
-elation with the desired degree of accuracy. 
Ii the correlation in the large range is known, 
wy estimate of the correlation in the small 
range may be had through the use of well 
»own formulas, provided the standard devi- 
tion of one of the variables in the small range 
is available. The partial correlation technique 
not applied here either because the number 
lasses in the third variable is usually small 

r because the regressions are not linear. 
Sometimes the situation is reversed. The 
rrelation of XY with Y within a grade or a 
ngle age class is known and it is desired to 
estimate the correlation in a range of several 
grades or age Classes. The usual procedure is 
make the necessary assumptions of linear- 
ty and homoscedasticity and apply one of 
the formulas available for the purpose, if the 
standard deviation of one of the variables in 

the large range is known. 

The writer has pointed out some of the 
ficulties encountered in applying the 
formulas for estimating the correlation in one 
range from that obtained in a different range 
when rectilinearity and homoscedasticity are 
not present in the data.’ In some cases the 
ibsence of these properties from the correla- 
tion chart may be due to the fact that the 
listribution of one variable in the large range 
s homogeneous while that of the other vari- 
ible is heterogeneous. As used here, the term 
homogeneity” implies that the means and 


Casanova, T. “A test of the assumptions of linearity and 
moscedasticity made in estimating the correlation in one 
— from that obtained in a different range.”’ J. Exp. Educ. 
VI 45-249, 1939. 


standard deviations are equal in all the grades 
or ages. This quality was observed while 


fi 


nding the correlation between an arithmetic 


test and the school marks in the subject. 
When the marks A, B, C, D and F are turned 
into numerical equivalents, the means of the 
several grades are approximately equal to each 
other, and the standard deviations are almost 
exactly equal, while the test distribution is 
heterogeneous throughout the grades. The 
same situation prevails whenever the same 
rating scale is used separately to classify the 
children in each grade. Thus, homogeneity is 
usually observed when each of several classes 
is rated on the basis of a second trait through 
an identical rating scale, and then the rec- 
ords are mingled into a single total group. 
However, this property may also result from 


d 


irect measurement, as may be observed in a 


chart showing the correlation between two 
traits within a given age range, one of which 
traits continues to grow throughout the en- 
tire age range, while growth in the other trait 
has ceased before reaching the lower limit of 
the age range, development in this trait re- 
maining practically stagnant throughout the 
range. In this case the means of the age sub- 
groups will increase with the age in the grow- 
ing trait, while the means and standard devia- 


ti 


ons of the stagnant trait will be approx- 


imately equal in all the subgroups. 


As far as the writer knows, none of the 


existing formulas is suitable for transmuting 
correlations from one range to another in the 
presence of the above described conditions. 
Let R,, be the correlation of X with Y ob- 
tained by mingling the records of m subgroups 
into a total single record. Then, we have,’ 


? For this formula in more explicit form, that is, without 


the summational notation, see Dunlap, J. Combinative prop- 
erties of correlation coefficients. J. Exp. Ed., 1937, V, 286- 
288. 


n 
S [Ms (re y.08.0%, + ds, dy] 
i= 
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,ay, and ry, y, are the statist- 
is the difference be- 


in which .V,, vo, 
ics of the subgroup 2, d, 
tween the mean of subgroup 7 for the x- 
variable and the mean of the total group for 
the same variable, and d,, is the correspond- 
ing difference in the y-variable. If the y- 
variable is homogeneous so that M,, == M,, = 
M, P : Oy, —— . and dy, * = 


d, a 0, 


[yc y Ox | 


d,')] | 


— —+7-(2) 


o 


where .V is the number of cases in the total 
group. This formula gives an estimate of the 
correlation that would be obtained by direct 
calculation in the large group when the sta- 
tistics of the component groups are known. 
Although it has no practical application be- 
cause the means of the x-arrays are not rec- 
tilinear and, therefore, the Pearson “r’’ is not 
a fair measure of the existing relationship, it 
may serve to detect the error made in esti- 
mating the correlation in the large range 
through formulas in common use. 

If the correlation in the total group is 
known the correlation in one of the sub- 
groups, or their average, may be estimated. 
The quantity in the bracket in the denom- 
inator is equal to S,, the standard deviation 
of the heterogeneous variable in the large 
range. Now, if all the o,’s are all equal to 
each other, or nearly so, and if V, — N, = 

N,, 


where 7, is the average correlation in the sub- 
groups. Since x is the heterogeneous variable, 
S,>o,, and r,>R,,. This is an exceptional 
result in view of the fact that correlations in 
wider ranges are usually expected to be larger, 
as evidenced by the formulas for their esti- 
mation in terms of correlations in narrow 
ranges. 
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In case that the x— distribution js th 

a test given in several grades, the differs, 
between the mean of one grade and tha: 
the next higher grade is, in genera]. no» 
constant. If this difference is assumed + 
constant, if its value is equal to A,, and jj + 
mean of the total group lies at the midpo» 
of the distribution of the means of the < 


groups, 


n(n- —1) 
——A 
IZ 


n 
> d.? 
1 I 
The fraction on the right is the well] k) 
formula for the sum of the squares \ 
deviations of the first m natural numbers 
around their own mean. The above equatiry 
is easily understood by supposing that ¢! 
mean of the first natural numbers has bee: 
subtracted from each one of them and eac! 
remainder multiplied by A,. Substituting ¢ 


n 
above value of yd) in formula (2) 
i I 
making again the assumptions involved in the 
derivation of (3), 


where once more it appears that r,>K 

the value of the fraction under the radical 
can not be negative. The value of r, increases 
in comparison to that of R,, as the common 
difference, A,, increases, and as the number 
of subgroups increases, but the proportion & 
independent of the unit of measurement oi * 
as this is a common factor of the numerator 
and denominator of the fraction under th 
radical. 

The preceding formulas compare the « 
relations in the subgroups with the corres 
tion in the total group when one of the vat 
ables is homogeneous. The most importan! 
problem, however, is to estimate the correl2- 
tion that would be obtained for the tota 
group if both variables were heterogencous 
from the correlation that has been observec 
when one of the variables is homogeneous 

Fig. 1, slab ABCD, shows three regress" 
lines for a hypothetical case in which y is the 
homogeneous variable and x is the heteroge 
neous variable. The lines represent the rege 
sions of y on x; and my,, m,, and my ue 
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Fig. 1. Shifting several regression lines into rectilinearity. 


‘he means of the subgroups 1, 2 and 3. Here 
”,, = My, == My, = M,, the mean of the total 

It is further assumed that m,, — 
mi, <= M, — My, = A,,, that A, is a positive 
yantity, that the slopes of the regression 
ines are nearly equal to each other, and that 
each regression is linear. 

One way of making rectilinear the data of 
the total group, and thus introducing heter- 
geneity in variable y, is by shifting distribu- 
tions 1 and 2 upward until their regression 
ines AL and OP interest line HJ respectively 
t points m', and m, , as shown in Fig. 1. 
let 6 stand for the slope of the line HJ. Then 
‘he distance moved by KL is m,, m',, = b A,, 


wd the distance moved by OP is mM = 
»\,. This amounts to adding the quantity 
4, to every y— measure in subgroup 2, and 
‘he quantity 26 A, to every y— measure of 
subgroup 3. But 


Bae Cu Kas 
h A. 7 x Oy 'xy 
Ox 


ind it is seen that the increments 5 A, and 
’ A, are independent of the unit of measure- 
‘nt of the x— variable. As the scales of 
‘easurement are purely arbitrary, the amount 


to be added to the y— measures of subgroups 
2 and 3 may be selected at will by choosing 
the proper scale for the y— variable. If all 
the measures are expressed in standard scores, 
b ==r,y, and the increase in y per unit © in- 
crease in x depends on the correlation alone. 

Suppose that x represents a test in a certain 
subject and y the school marks in the same 
subject for a group including various grades. 
If the same rating scale, say A, B, C, D and 
F, is used in all the grades, the y— distribu 
tions will be homogeneous, and the means 2nd 
the regression lines will lie as shown in Fig. 1. 
The numerical equivalents assigned to the 
letter-grades are at our disposal, and so is 
A,, the quantity to be added to every measure 
of group 2. But if the latter is expressed in 
terms of oy, it will be independent of the 
numerical equivalents, for if these are multi- 
plied by a constant &, o, will be also multi- 
plied by &. 

In the case of m subgroups, the effect of 
shifting the regression lines, that is, the effect 
of adding the constants 5 A,, 2b A, 

(nm —1)b A, to each of the y— measures of 
the groups 2, 3, n respectively, will be: 
1—The y— means of the subgroups 2, 

n will be respectively increased by } A,, 
(nm —1)bA,. 
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The «,’s will remain unchanged, as add- Let 

ing a constant to each measure does not affect 
the standard deviation. 

3—-None of the statistics of the x— vari- 
able alone will suffer any change no matter 
what the distance moved in each case. But Then. 
the means and standard deviations of the y- 
arrays will be affected as these are functions 
of both variables. 

4—-The correlations within the subgroups 
will not be altered by the addition of a con- 
stant 
5—Since A, is now equal to 6 Ay, using (5), 


5 
the last term in the numerator of (1) becomes 





nin I) . 
bh A* 


12 x 
6—-In like manner, the second bracket in 
the denominator of (1) is now 


i 
n(n I 4 
bh? a? » 


I y\ 


255 If x and y are expressed in terms 
Formula (3) gives the value of the correla- ard scores in the subgroups, 
tion observed in the total group, R,,. Let 
c R,, be the correlation in the total group 
after the above corrections for homogeneity , , 
have been made. Then the equivalent of R,, Ry, 
in (3), after correction, is 


, “th Now 6 ==r,,, the correlation in subgroup | 
(w — 4) 4, Therefore, 


I20 
y 


+ 


2 
-(7) —- > Om 
(n 1) A. b ot, 


S.4/ 14 
| I20, 


2 
y 


In (g) it may be seen that if all thers 
the equivalent of (4), after correction, is have the same sign, } and r, are of the same 
. (n? sign, and that o,, must be positive, since ! 

03%, — is involved in the expression. Therefore, the 

¢ Rk, = ———— .. (8) fraction in the last equation is a positive in- 
(n? —1) a, 6? proper fraction, and C and R,, have the same 

Ss ] i i I2o. sign, that is, the correction for homogene'ty 

F always increases the absolute value of th 

observed correlation, whether its sign is pest 


and the correction for homogeneity, C, 
i tive or negative. 


is equal to ¢ Ryy — Ryy, is 
(m?@—1) 3° b i. 


I20.° 
y 
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ymulas were applied to the correla- 

of an arithmetic test and the school 
293 children of 6th, 7th and 8th 

lhe school marks of A, B, C, D, and 
ssigned equivalents of 4, 3, 2, 1 and 

i the records of the last three years were 


Here the average difference in the m,’s was 
put in the place of A,. This result agrees with 
that obtained through formula (3), but both 
fall short of the actual value which is .41. 
This is due to the assumption that the m,’s 
are equal, which results in eliminating the last 


‘ied to give a range of from 1 to 12. The 
satistics of the school marks and the test 
as follows: 


term of the numerator of (1), thus leading 
to a low estimate. Substituting the average of 
the o,’s in (8), we have 


116.21 X .10 
12 X 5.11 


“116, 21 —_ 


x 110.04 X «| 
12 AF 


< “110.04 > X Or 


I2 X §.11 


GRADE 
Sti itistic® 7th 8th All 
ranew O26 6.293 0.47 6.27 
«nn O29 2.30 
— 2.20 
77.61 
13.23 


and by (ga) 


x ---56.06 
aaah EFS 


The observed value of R, 
(ga). If instead, the value of k,, calculated 
by formula (6) is substituted, C — .07 
agrees with the values of R,, 
.44, as obtained above. 


—— was substituted in 
— 114 
™ ™ , which 
———— . 07 37 and ¢ Ryy 
By formula (3) 

i -45 X_ 12.50 


= .37 Formulas (9) and (ga) are the most con- 
15.28 ; 


’ venient and accurate to use, since the error 
The average of the o,’s was 
oe made in assuming that m, == my = 
lace of o,. By formula (6), 
45 m,.., which resulted in the elimination of the 
f+ ee 


put in the 








8 X 116.21 = 37 last term of the numerator of (1), is not 
y 12 X 156.25 present here, because this term is eliminated 
‘The statistics were first calculated to four decimal places. when R,, is subtracted from Res. 








EXPERIMENTAL DESIGN AND STATISTICAL TREATMENT 
IN EDUCATIONAL RESEARCH 


EUGENE SHEN 
Harvard University 


I 


Experiment distinguishes itself from other 
types of observation by the presence of some 
control of the conditions under which obser- 
vations are made. Curiously, the experimen- 
tal factor which is a variable under direct 
control is conventionally never referred to as 
a controlled variable. On the contrary, con- 
trol in the latter term means holding con- 
stant, so that a controlled variable is precisely 
a factor which is not allowed to vary. 


In educational research, direct regulation is 
feasible for only a few of the many factors 
that require control. Most obviously, the 
heredity and previous experience of the re- 
agents are variable beyond the experimenter’s, 
or any human, power, and such variables can 
at best be brought under indirect control by 
means of selection in the form of matching 
between subjects with reference to some 
measurable character. But equating is not 
always feasible between groups receiving dif- 
ferent experimental treatments, and is never 
desirable within such groups. A still more 
indirect method is then substituted whereby 
the differences are merely noted and their 
effects allowed for in the subsequent evalu- 
ation of results. Such adjustment is termed 
statistical, as distinguished from experimental, 
control. 

When all these various methods of control, 
indirect as well as direct, statistical as well 
as experimental, are resorted to, there are 
always other variables which must be left to 
chance. The very necessity of using some sta- 
tistical tests of significance in the evaluation 
of experimental data speaks for the omni- 
presence of such uncontrolled variables. 
While increased control generally entails im- 


1Some direct control is generally implied in the term ex- 
periment. A study of sex differences, for instance, becomes 
experimental only when the character of sex itself is experi- 
mentally intemal, or when the two sexes live under experi- 
mentally arranged conditions, or at least when they are 
experimentally assigned to comparable existing conditions. 
Merely matching the groups with respect to biological and 
social background, however adequate for given purposes, is 
not generally accepted as experimental procedure. This con- 
— distinction is not essential in the following discussion, 
owever. 
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proved precision whereby an experiment § 
rendered more efficient, the presence of ever 
large chance factors does not necessar 
vitiate the validity of a conclusion, provi 
the proper test of significance is salons 
Indeed, the evaluation of the st: andard error 
is logically nothing but a case of statisticy 
control for the random effects of unmeasured 
factors. 
The proper evaluation of observed differ. 
ences between groups receiving 
treatments in the light of variations within 
groups of subjects uniformly treated is based 
upon an assumption of the unmeasured var 
ables being randomized with respect to the 
experimental treatments. If an assumed) 
random factor operates in a biased manner 
any computation of standard error will be 
idle and misleading, unless the direction and 
amount of bias can be determined and allowed 
for. If, on the other hand, a controlled vari- 
able is treated as if a random variation were 
permitted between the groups, then the sig- 
nificance of the difference will be underesti- 
mated. Such invalid estimation of significance 
is a necessary consequence of incoordination 
between initial design and subsequent treat- 
ment, or a failure to grasp as a unity the dil- 
ferent phases of a coherent logical structure 


diffe rant 


II 


The most general type of educational pro} 
lem is limited to two experimental treal- 
ments; two methods of instruction in a cer 
tain subject, say. Aside from a minimum 
direct control of some factors, th 


¢ 


such as t! 
length and distribution of instruction perio“ 
which will always be necessary, the fe , 
which other factors, such as the maturity anc 
brightness of the pupils, are subject to match- 
ing and statistical control, is left to the opt mn 
of the investigator. The crudest experimental 
design will leave all variations between pup! 
uncontrolled, and the corresponding statistical 
treatment is then a computation of a critical 


ratio of the following form, 





EXPERIMENTAL DESIGN 





a ee 


S(x,—x,)? 





here x, and x, refer to individual scores on 

' achievement, x, and x, to their means, 
, and «, to the standard deviations, and N, 
and. to the number of pupils, respectively, 
‘the two groups. The interpretation of this 
ratio is made in terms of the probability of 
‘he observed difference arising out of chance 
variation from an hypothetical difference of 
vero, usually by referring to a table of the 
normal distribution. A slightly different treat- 
ment, which is not yet in general use in the 
jeld of education, is the ¢ test, based on the 
following ratio, 


x,—X, 


N 


biased view is that the more extensively used 
method is a crude approximation never per- 
missible except when samples are large. 
However, a much more serious question 
with reference to the usefulness of either 
formula is whether the uncontrolled variables 
are really randomized. In a laboratory ex- 
periment a randomized assignment of each 
subject to either group can be executed not 
only with ease but with perfect confidence. 
In classroom experiments, on the other hand, 
groups organized upon principles quite con- 
trary to randomization usually have to be 





OP negicentientabcuanminatpealbcsnia [2] 





S(x, —x,)? + X(x, =2)/ I 
N, + N,—2 


The ¢ test differs from the prevalent 
method in three respects: (1) The variance 
within groups is given a single determination 


rather than separately estimated for each 


group. Since the very hypothesis to be tested 
wstulates the two groups as being random 
amples of the same population, a separate 
estimation of variance within each group is 


learly unnecessary. (2) In computing the 
variance within groups, the number of de- 
grees of freedom is used in the denominator 
rather than the number of observations. This 
s necessitated by the fact that deviations 
from an observed mean are on the whole 
aways less than deviations from the true 
mean, and it can be shown that this discrep- 
ancy is properly corrected for by using the 
number of degrees of freedom, which is the 
number of observations minus the number of 

mstants derived. (3) The significance of ¢ 
‘ interpreted by referring to a table of the 
‘ distribution, which approximates the nor- 
mal only as the number of degrees of freedom 
s indefinitely increased. 

Since in practical importance the difference 
between the two methods varies inversely as 
the size of the samples, those accustomed to 
the less accurate procedure are likely to re- 
gard the ¢ test as an over-refinement useful 
nly when samples are small, whereas an un- 


«For a full treatment of the ¢ test, see Fisher, R. A., Sta- 
“wer ag Methods for Research Workers. London: Oliver and 


, 


N, 


+) 
N. 


kept intact, and consequently the fundamen- 
tal assumption involved in the use of either 
[1] or [2] hardly ever holds. The objection 
to the use of supposedly random groups, that 
the standard error would be left at a max- 
imum, is therefore of rather secondary impor- 
tance, the primary consideration being 
whether any test of significance is valid at all. 


III 


An improved experimental design, and one 
much more generally employed and trusted, 
is that of groups equated by selection on the 
basis of some measured character or char- 
acters which otherwise would disturb the 
effects of the experimental variable. Variables 
most often used as bases of matching are, 
either jointly or separately, age, intelligence, 
and initial status in the field of learning un- 
der experiment. Two groups are usually said 
to be equated when both have the same mean 
and variance. Sometimes, however, equating 
is more meticulously achieved by pairing, 
each to each, the individuals of the two 
groups. Provided that the matched characters 
are closely related to the final achievement 
upon which the relative efficiency of the two 
instructional methods is to be judged, either 
method of equating will remove bias between 
the groups and reduce the actual error vari- 
ance to a considerable extent. 
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The adoption of a design which does away Earlier in the discussion it was ment 
with portions of the real errors in the experi- without argument that equating within gy 
ment, be it noted, must be accompanied by _ is never desirable. Such equating, while ¢; 
their elimination in the statistical treatment tive in minimizing the standard error. 
of results. Unless this is realized, the in- at the same time minimize the generaljy, 
creased precision becomes worse than useless, the conclusion. In other words, the 
since, in being inevitably thrown away by an _ increase in the significance of resu Its. wou 
inappropriate test of significance, it impairs be accompanied by a corresponding decres: 
the validity of the conclusion.’ in the range of their applicability, 7), 

For the method of pairing, a correct test method of pairing is powerful to the exter 
of significance in place of [1] or [2] is not it is able to reconcile the apparently cont; 
difficult to devise and has indeed been rather dictory requirements of a large variabilir 
extensively used. One needs only to compute within groups and a small standard error ¢ 
the difference for each pair, make a distribu- difference between them, by a high corr 
tion of such differences, and obtain the mean tion between members of pairs. 
and its standard error in the usual manner. The less meticulous method of matching 
The critical ratio is thus groups as wholes is in fact as effective as ; 


Xx, Xv 


oO 


where x,, ¥., %,, ¥., #, and #, have the usual more laborious method of pairing. But owing 
meanings as in [1] and [2], 7,. designates to the frequent use of wrong tests of signi 


the correlation between the two groups on cance, its advantages have not been so obvyi- 
final achievement, and V the number of pairs. ous. Usually formula [1] has been used 


The corresponding ¢ test, applicable to small default. 


as well as large samples, is Though formula [3] appears in: ogy 
when groups are matched as wholes, i 
occasionally been used. The pears 
sists in deriving regression coefficients 
final achievement upon the matched variabl 
or variables, employing the regression equa- 
tion to make best estimates of final achieve. 
It is interesting to note that the matched ment for each subject, ranking the subject 
variables do not enter in either [3] or [4], of each group in the order of their estim 
from which it may appear as if the thorough- scores, pairing the individuals of the 
ness with which the subjects were paired groups according to their rank order, using 
would be of no consequence. But it is clear some irdividuals more than once if the tw 
that the effectiveness of pairing is propor- groups are unequal in number. and fina 
tional to the size of the correlation coefficient, computing the correlation of obse rved final 
which attains a2 maximum only when all the cores on the basis of such pairing. Whik 
relevant factors are used in pairing. It is un- _ pecessarily a makeshift which is at once in 
necessary to mention the obvious fallacy of elegant and laborious, this method is esser- 
pairing the individuals on the basis of rank  ¢jally sound and is to be preferred to a neat 
order without equating the groups. misapplication of the random-group formu’ 


‘It is sometimes maintained that errors of rejecting sig- . = = , . 
nificant differences as insignificant are less serious than those But a_ formula especially ¢ derived 
4% accepting as significant differences which are really insig- matched groups is available.’ Since the vari 
nificant. This seems to have been based upon an ill ¢ dnceived 

pplication of the principle of parsimony. Two fallacies are ation of final scores which is due to variat! 


involved. In the first place, inasmuch as experiments may be o}jminat ted 
and often are designed for the confirmation of the obvious as in the matched factors has been elimi 

well as the exploration of innovations, the direction of con- between groups, by experiment: il contro! 
servatism does not always coincide with either tpye of error 


Then, what is more important, the uncritical application of is only logical also to eliminate such vi ariatio 


an over- or under-estimated standard error has consequences 1 Psychol 
quite other than those of a voluntary adoption of a higher _ See Wilks, S. S., Journal of Bivens | y07-208 
© lower level of significance XXII: 205-208. Also Lindquist, E. F., ibid, 19/-- 


[4] 
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groups, by statistical control. The re- 
critical ratio is 


hin 
Lillis 


sftin 
sullil 


’ 


'_nn)t+=—(1t — Vx) 
N, 

vhere the r’s refer to the correlations between 
.e matched variable and final achievement 
res, for each of the two groups, and the 
er symbols have the same meaning as be- 
‘ore. The relation between [3] and [5] is 
sily seen if we remember that any correla- 
n between final scores of the paired indi- 
ls is always due to a correlation of each 
In other words, 


id 


» the matched variable. 


1.00 and 7 == 00 


lx.xe 


follows that 


Vxyx2 = Taiy1 + Txoy2 


Matching is thus seen to have the same power 
s pairing, in rendering the experiment more 
efficient, with the further advantage of rel- 
tive simplicity in practical procedure. 


In the light of discussions of formula [2], 
minor modification of formula [5] will re- 
luce it to the form of ¢, 


where a single correlation coefficient is deter- 
mined for both groups combined, and the 
number of degrees of freedom is one less than 
n formula {2], since an additional constant, 
r, is computed. 

If two or more characters are used as the 
jases of matching, formulae [5] and [6] can 
ve generalized by using multiple as well as 
imple correlation coefficients, the number of 
legrees of freedom in [6] to be reduced by 
ne for every additional variable used in 
matching, 


IV 


The discussion so far has been limited to 
te case of two variants of a single experi- 
mental factor. However, it is often con- 


- While this formula and formulae [7] and [8] may appear 
* they are merely applications of Fisher’s methods. No 


' will therefore be given. 
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venient, if not necessary, to study in a single 
set-up two or more experimental factors each 
of which may, further, have more than two 
variants. An investigation of the distribution 
of practice, for instance, will probably have 
as experimental variables both the length of 
practice periods and the length of interval 
between them. Since all lengths of practice 
will be assigned to each length of interval, 
the fundamental principle of experimentation, 
that only a single factor should vary at a time, 
is not violated. On the contrary, such a com- 
prehensive design, besides being more econom- 
ical than separate experiments on the two 
factors, makes possible a study of interaction 
between them, which in this instance is un- 
doubtedly very important. 

The statistical treatment most appropriate 
to such experimental designs is the analysis 
of variance. While a full treatment of the 
subject is beyond the scope of the present 
discussion, the method is applicable to so 
many situations, and the concept is so help- 
ful in selecting correct tests of significance, 
that the fundamental ideas will be briefly 
presented.? 


The total variance of a whole series of 
observations is usually divisible into a num- 
ber of independent portions, at least to one 
portion which is the variance between the 


" [6] 


I 
a 


means of groups, and another which is that 
within groups. If there are two or more ex- 
perimental factors, or if a single factor has 
more than two variants, the variance between 
groups is further analyzable. Likewise, if 
there is matching or pairing between groups, 
the variance within groups is further analyz- 
able. Any two portions of variance can now 
be compared with each other, but the most 
important comparison is always with the 
residual variance ascribable to error. 


’ I 
xy) NV 


Let us consider the simplest case of two 
experimental treatments. The total variance 
is 


3(x, — x)? + 3(x, — x)? 


1The reader is again referred to Fisher. Besides his Sta- 
tistical Methods for Research Workers, see The Design of 
Experiments, London: Oliver and Boyd, 1937. 
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to be divided by V, + N, — 1 degrees of 
freedom. This is analyzable into two portions, 
the variance between groups, 

N(x 


1 1 


ee I I 
(x, —*2)"/\ NTN, 


x)? + N.(%. x): 


with a single degree of freedom, and the 
variance within groups, 
S(x,—x,)? + B(x, — 2.)? 


to be divided by V, + N, — 2 degrees of 
freedom. 

If the two groups are random samples of 
the same population, the whole variance with- 
in groups will be ascribable to error. If the 
method of pairing is used, then the variance 


between the means of pairs, 


3(x,—x, + x, —2,)? 


to be divided by N — 1 degrees of freedom, 
should be eliminated from the variance within 
groups, and the residual, 

1—%,—%, +3)" /2 

to be divided by NV — 1 degrees of freedom, 
is the correct estimate of error. If matched 
groups are used, then a portion of the variance 
within groups which is ascribable to varia- 
tions in the matched variable, 


ats 


[ S(x, x,)? + 3(*, — x,)?] rey 


should be eliminated, and the residual, 


X(x, — x,)?] [1 — ry] 


to be divided by V, + N, — 
freedom, is the error variance. 

The reader will have observed that when 
the variance between groups is divided by the 
appropriate error variance, the resulting 
ratio is exactly equal to the square of ¢ as 
defined in [2], [4], and [6]. The ¢ test is 
thus a special case of the almost universally 
applicable method of analysis of variance. It 
also becomes obvious, since any two portions 
of variance may be compared with each other, 
that wrong tests of significance can be cor- 
rectly interpreted. If [2] is used where [4] 
or [6] should be, the real question being 
tested becomes, not whether the experimental 
treatments are more important than chance 
variations, but whether they are more impor- 
tant than individual differences plus chance 
errors. 


[X(x, x,)? + 


3 degrees of 
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The method of analyzing variance Within 


Bu 


ph experimental designs. The rec 
difference lies in the greater number of groys 
and more complex relations among them, |: 
one employs, in the problem on distri}, 
of practice, four different lengths of practic. 
periods each with four different  interyak 
there will be sixteen different treatments with 
fifteen degrees of freedom, or fifteen inde le. 
pendent comparisons among sixteen grou 
The variance between groups can be divi ied 
as follows: 
Degrees f 

Source of Variation Freedom 
Between lengths of practice periods 
Between lengths of intervals ____ 3 
Interaction between two factors - 4 


Total variance between groups - 15 


Each of these three portions may be fur 
ther analyzed. One possible subdivision « 


of practice psorang or between pico of in- 
tervals, is a comparison of the frst tw 
lengths with the last two, a second comparison 
of the first and third lengths with the second 
and fourth, and a third comparison of th 
first and fourth lengths with the second ar 
third. A more significant treatment is to ft 
a regression line or curve to the four means 
and divide the variance into a portion due t 
regression and another to deviation from th 
regression curve. 

The interaction variance can likewise | 
further analyzed. In the above problem. 
would not expect the effect of different pra 
tice periods to be independent of the difteren' 
intervals between them; i.e., at least some 
the nine independent comparisons would shov 
significant differences. If interaction is insiz 
nificant, that portion of variance may be use‘ 
for an estimate of error. Therefore, when 1 
teraction can be ruled out on a priori grounds 
an experiment may sometimes save the !a! 
and expense of replication and stil] have 
valid estimate of error.’ 


Vv 

Associated with the analysis of variance 
the analysis of covariance, or the determin 
tion of regression, the purpose of which 's‘ 


evaluate the effects of concomitant variables 


1 Crutchfield has proposed to use in an experiment /*° ' 
no two of which are to be treated alike. See /°#"" 
Psychology, 1938, V: 339-346. 
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‘that they can be eliminated. Formula [6] For the case of a single concomitant vari- 
- in fact made use of analysis of covariance, able, it is obvious that the required adjust- 

by which the proper error variance for ment is of the type 

natched groups is obtained. But the method (xy)? / Sy? = 62, Sy? = 73, Bx? 

often used in situations where a statistical 

adjustment is required for the variance be- where, to be consistent with the preceding 

-ween, as well as within, groups, i.e., where symbols, x refers to final scores and y to a 

nerfect equating between groups is not feas- concomitant variable. In the case of two 


ble. Those who have experience with match- groups, the adjusted total variance is thus 





a(x, —3)° + 10, —3 — FE 9M te 
=(y, — vy)? + S(y. — y)? 








ag groups will have found, especially when to be divided by V, + N, — 2 degrees of 
‘wo or more characters are used in matching, freedom, while the adjusted variance within 
that it often necessitates repeated trial and groups is 





[=(x, sea x,)(V; —7) — = (x, —*,) (2 — y2)]? 
=(¥, — ¥,)° ao =(¥. nahi ¥2)? 





3(x, —%,)? + 3(x, — x)? — 





error, and considerable loss in the total num- to be divided by N, + N, — 3 degrees of 
ver of cases. They will thus have doubted as freedom. The variance between groups cannot 
‘) whether an experiment may not be ren- be directly adjusted, but the required result 
ered more efficient by retaining more cases _ is obtained by subtracting the adjusted vari- 
ind allowing for slight inequalities between ance within groups from the adjusted total 
ips.' Moreover, there are concomitant variance, thus yielding 


Uy} 





[3(x, — x) (y, — y) + 3(x, — 2%) (92 —y)]?_ 
X(y, — ¥)? + 3(¥, — ¥)? 





N, (x, — x)? + N,(z,— x)? — 


[3(x, — x) (9, — 1) + 3(%, — 82) (2 — ¥2) J? 
=(y, + ie y,)? + =(¥, 2 niall ¥)? 








variables which may not be desirable to This divided by the adjusted variance within 


equate under certain circumstances. A tech- . . : 2 whi 
sique for the adjustment of inequality be- groups is the variance ratio or ¢*, which can 
tween groups is thus very useful. be reduced tc the following form:' 





ae [ (x, — x.) — d(y, —-¥.)]}? 
> (x, —x,)? tise * [3(¥, — y,)? + 3092 — 2)? at 
NV, + N, = J 
he (y1 — 9a)* Vtg] 
VN 30, — yi)? + 302 — 92)? S 














S(x, = 1) (1 — 4) + (Ha — ¥) (Ye — Je) 
- >(¥, — 91)? + 3(¥2 — 92)? 


. a ractical difficulties of matching will be much re- 1 As stated before, this is merely an application of Fisher's 

ea if the means are equated without also equating the methods, even though s * never gives such an explicit 

‘sdard deviations, which is far less essential. formula. Cf. Goulden, C. H. Methods of Statistical Analysis. 
New York: John Wiley, i030, pp. 250-253. 














An extension of formula |7]| for the case 
of two or more concomitant variables would 
make it unmanageable and is therefore in- 
feasible. But a generalization becomes at the 
same time a simplification if stated in terms 
of the correlation between final achievement 
and concomitant measurement. The result is 
formula | 8]: 


where KX is the correlation with mean of the 
total as origin and r with the group means as 
wrigin. The correlations may be multiple or 
simple, and & is the number of concomitant 
measurements made.’ 

For the case of two groups, to which most 
educational experiments seem to have limited 
themselves, formula [8] gives a most gener- 
ally applicable test of significance. If the 
groups are assumed to be random samples 
without concomitant measurement, then R, r 
and & will all be zero, and formula [8] re- 
duces to formula [2]. If the groups are 
equated, then R attains a minimum relative 
to r, and formula [8] reduces to formula [6]. 
Slight inequalities between groups will be 
properly allowed for in the value of R, which 
varies as a function of such discrepancies. 
rhe size of r is an indication of the degree 
of adequacy with which relevant concomitant 
measurements are made. If a perfect r could 
be obtained—without using so many variables 
as to reduce the number of degrees of free- 
dom to zero, and without permitting such ex- 
treme inequality between the groups as to 
render R equal to r—it would mean a perfect 
control of the subjects, and further statistical 
tests of significance would be unnecessary, 
since any difference would then be signifi- 
cant.' 

Formula [8] shows that there are five 
factors which may influence the significance 
of a given difference: r, R, &, and the two 
V's. The greatest significance is achieved by 
minimizing R and &, and maximizing r and 


‘Formulae [7] and [8] have been discussed somewhat 
more fully in the January, 1940, issue of the Harvard Edu- 
cational Review, with a numerical illustration by lf. & 
Deemer. 


*Experimenters in laboratories who do not bother about 
statistical tests of significance unwittingly assume this ideal 
state of affairs, which is probably never attainable in the 
field of learning 
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x, —x,)? (1 - 


\Vo 


the two N’s. The practical problem {or ; 
investigator is two-fold: First, in order ; 
maximize r, he should make as many 
concomitant measurements as feasi}! 
point is usually soon reached after 
increase in the multiple correlation is s 
that it will not be worth the ac 
increase in &. Second, in order to mini 


kR? 


- X(x, — x.)?] 


) 

I I ] 
N.S 
R, relative to r, the two groups sh 
matched so that the means of the o 
measurements are as nearly equal as | 
This, however, can only be achiev 
ducing one or both of the two .V’ ly 
discrepancies are only slight, further appr 
imation to perfect equality is usually not 
difficult, but detrimental. Another p 
should be noted is that an equal divis 
the total number of cases between the t 
groups gives a maximum efficiency 
experiment. 

VI 

There remains the consideration that 
gression of final achievement upon the 
comitant variables may be different 
different experimental treatments. This net 
not be assumed in usual experiments w! 
are limited to relatively homogeneous ¢ 
of subjects. It is logical to expect, howe 
that methods which are suitable to younge: 
children need not be so to older ones, t! 
treatments which are very helpful to the 
may even prove a handicap to the bright 
that a new system of arithmetic or handwr! 
ing which would be distinctly superior 
adopted from the beginning, will probably 
terfere with habits in some pupils who bh 
begun by the old method. When subjects " 
clude a wide range of maturity, brightness 
initial status of learning, therefore, it 
always be instructive not only to determin 
the regression equations separately for ( 
groups, but, further, to study the rang 
significance in terms of the concomitant var 
ables. In other words, the concomitant meas 
urements would then, instead of being 
trolled factors, become experimental varia’ 
in addition to the major factor of the direct!) 
regulated treatments. If the correspon 
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coefficients differ considerably be- 
» the two groups, it will not be surprising 
ne of the two treatments superior in 
and inferior in another, while a test 
is only concerned with averages may 

| significant difference at all. 
he statistical analysis for such problems 
necessarily be somewhat more compli- 
But the required analysis of covari- 
ind adjustment of variance will involve 
ew principle and should occasion no 
; difficulty. To test the significance of 


differences between the experimental treat- 
ments for all values of the concomitant vari- 
able or variables, it is only necessary to set 
t appropriate to any chosen level of signifi- 
cance and solve for the limiting values or 
boundary curves in terms of the concomitant 
variables. 

*Cf. Johnson, P. O., and Neyman, J., “Tests of Certain 
Linear Hypotheses and their Application to some Educational 
Problems", Statistical Research Memoirs, 1936. Vol. I, pp 
57-93. Although these authors make use of a different test of 
significance, the concept of a ‘region of significance’ applies 
to the ¢ test equally well 











PERSISTENCE OF ATTITUDES CONCERNING 
CONSERVATION ISSUES’ 


A. C. WILLIAMSON AND H. H. REMMERS 


I. THe PROBLEM 


Ihe problem of this study was threefold: 
firstly, to measure persistence of group atti- 
tudes changed by means of defined social 
stimulus material, secondly, to measure the 
respective variabilities of the groups to dis- 
cover whether they become more or less 
homogeneous after the presentation of such 
material and thirdly, to compare rural with 
urban group-attitudes produced by defined 
social stimulus material and to compare their 
relative homogeneity after the presentation of 
such material. 


II. PROCEDURE 


Four equivalent forms, W, X, Y and Z, of 
the Thomas—Remmers generalized attitude 
scale, Forms A and B to Measure Attitudes 
toward Any Proposed Social Action were de- 
rived by a “split-half” technique, whereby 
Forms A and B were divided equivalently and 
recombined in four different, comparable com- 
binations. Each new form was used to 
measure attitudes toward the following five 
attitude objects. 


1. Allowing the government to tell the 
farmer how to farm. 

2. Allowing each farmer to farm as he 
pleases. 

3. Clean farming. 

4. Taxing all the people to plant new 
forests, and 

5. Draining swamps. 


The scales were administered before any 
stimulus material was read, to 301 rural and 
urban high school pupils in the middlewest. 
The pupils in each high school were divided 
into four randomly selected groups of approx- 
imately equal number and each group was 
measured with a different form (either W, X, 
Y or Z) of the scale. 


‘Tor considerable aid in preparing the poms y for pub- 
lication we are indebted to Mr. N. T. Schmalzried, Assistant 
in the Division of Educational Reference. 


_ McConnell, Robert, “Attitudes Toward Certain Proposed 
Social Actions as Affected by Defined Educational Content,” 
Bulletin of Purdue University, Studies in Higher Education 
XXXI, Further Studies in Attitudes, Series 11, December, 
1936, pp. 100-104. 
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Stimulus material,? intended to chan» 
their attitudes in a given direction, i.e., {ayo 
ably toward attitude objects 1, 4 and unix 
orably toward objects 2, 3 and 5, was rea; 
on September 1 and the same students marke: 
a different form of the scales at intervals 
one day (post-test), one month (October 
test), four months (January test) and eigh 
months (May test, marked by rural grou 
only) after the pre-test. | 






















III. ResuLts 


The premeditated shifts’ in average atti. 
tude were accomplished in the case of each 
of five issues (Table I). These shifts were 
statistically significant and remained so {or 
each group on each attitude object, with 
three exceptions (Table IT), viz., (1) the 
difference between the pre-test and the May 
test of the rural group for attitude object one 
was 1.97 times its standard error: (2) 1! 
difference between the pre-test and the Jan- 
uary test of the rural group for attitude object 
2 was not statistically significant; (3) the 
difference between the pre-test and the Janu- 
ary test of the urban group for attitude ob- 
ject 4 was 1.32 times its standard error. The 
only statistically significant shift in average 
attitude, made by the urban group, which dic 
not persist as changed was for attitude object 
4, taxing all of the people to plant new forests 

Combining the results obtained from the 
rural and urban groups with reference to per- 
sistence of changed attitudes (Table II an: 
Figures 1, 2, 3, 4, and 5) one observes that 4 
statistically significant difference in attituce 
is maintained from the post-test to the Janv- 
ary test for each attitude object considere¢. 
The differences in variability within the tot 
group on the pre-test and subsequent tests 
are generally in the direction of a statistically 
significantly greater variability on post-tests 
(Table III). Important exceptions are ‘0 
attitude object one, allowing the government 
to tell the farmer how to farm, for which te 
difference in standard deviations between 


* The Remmers generalized scale items are arranged 0 & 
scending order of scale value. A higher value therefore me 
a more favorable attitude. 
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TABLE II 
DIFFERENCES OF MEANS FOR PRE-TEST AND JANUARY TEST 
Attitude 
Object Rural Urban Total 
Diff. g Cc. RB. Diff. o C. R. Diff. og ( 
diff. diff. diff 
1.29 35 3. 69 2.67 . 46 5.74 1. 86 24 7 7: 
l 
cy be 36 1.97 
84 37 ey 2.15 . 40 5.38 1.38 23 
7 75° 37 2.03 
1.11 20 5. 55 1.61 30 5. 37 1.31 16 
1. Ze" 21 5. 71 
1. 32 36 3. 67 .49 Dy 1. 32 . 82 19 { 
4 ™ 
1.25° 37 3.38 
1. 22 25 4.88 3.85 38 10.13 2.40 29 10.9 
» 
ey ig .25 4.68 
*Differences between Pre-test and May Test. 
1S. —_— Entire Croup 
oonesecces Rural 
14, ome oe «OU ban 
13 
a 
e& 
= 
4 
oa 
> 4 = 
@ 
9 
”v me. 
lL hie 
10 LL 
G 
— 
~_ 
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Pre-test 1 day 1 month 4 months 8 months 
(Post-test) (October test) (January test) (May test 





Ficure I 


Persistence of Attitude—Allowing the Government to Tell the Farmer How to Farm 





PERSISTENCE OF ATTITUDES 357 


Entire Group 





l 1 1 J 1 


Pre-test 1 day 1 month 4 months 8 months 
(Post-test) (October test) (January test) (May test) 





FIGURE II 
Persistence of Attitudes—Allow the Farmer to Farm as He Pleases 
Entire Group 
owvececeees Rural 


—_——_omae Urban 





y 
L | 1 a 1 ] 


Pre-test 1 day 1 month 4 months & months 
(Post-test) (October test) (January test) (May test) 
FiGcure III 
Persistence of Attitudes—Clean Farming 
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FIGURE IV 
Persistence of Attitudes—Taxing All of the People to Plant New Forests 
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FIGURE V 
Persistence of Attitudes—Draining Swamps 





arch, 1940] PERSISTENCE OF ATTITUDES 359 
TABLE III 
DIFFERENCES IN STANDARD DEVIATIONS FOR ENTIRE GROUP 
ttitu de 
at Post-test October test January test 
is Diff. o C.R. Diff. a C.R. Diff. o C: R. 
diff. diff. diff. 
4 A A ’ 
. 24 .18 1.33 .49 . 2. 58 53 .20 2.65 
A A A 
v4 . 92 .19 4.84 95 .19 5. 00 i | 18 4.28 
A A A 
2.52 16 15. 75 1.92 14 13.71 1. 47 12 12.25 
F < < < 
- 4 . 62 14 4.43 15 .15 1.00 .04 16 . 25 
E “ A * 
~ 5 2.55 18 14.17 2.19 17 12. 88 1.79 16 11.19 


Note: The arrow in each case points to the test which had the larger sigma. 


ore-test and post-test are insignificant (81 
hances in 100 of a difference greater than 
vero). These chances, however, increase to- 
ward a statistically significant difference to 
in 100 for both the October and January 


Bs 


, 


r attitude object 4, taxing all the people 
plant new forests, there is a statistically 


significant decrease in variability between the 
-test and the post-test. However, this dif- 
erence rapidly disappears on the October and 
January tests, the chances for a difference 
reater than zero going from 75 in roo to 
in 100 during the period from October to 
January. 
Rural-urban comparisons disclose statisti- 
lly significant differences in attitude on 
h attitude object (Table IV). These dif- 
ferences are present only once on both the 
pre-test and the post-test, viz., attitude ob- 
ect 4, taxing all the people to plant new 
ests, 
For attitude object 1 (allowing the govern- 
ment to tell the farmer how to farm) one 
serves no difference in the attitude of the 
‘wo groups on the pre-test, but after admin- 
stration of the stimulus material, designed to 
shift their attitudes in a direction favoring 
government intervention, the urban group be- 
omes significantly more disposed to allow 
‘he government to tell their rural neighbors 
how to operate farms. There are no statisti- 
‘ally significant differences in the variability 
if the two groups. 
The urban population proved to be sig- 
nificantly more in favor of allowing the 
‘armer to farm as he pleased, (attitude ob- 
ject 2) than did the farm children themselves 


on the pre-test, but this difference vanished 
after the stimulus material was administered. 
The two groups were practically the same in 
variability. 

The two groups were practically of the 
same attitude on the pre-test with reference 
to clean farming (attitude object 3), but the 
stimulus material caused the rural group to 
become significantly more unfavorable to- 
ward it than the urban group. The urban 
group was less homogeneous after the presen- 
tation of the stimulus material, their stand- 
ard deviations being significantly larger than 
the rural group for post-test, October and 
January tests. This corroborates previous 
findings that with exposure to information 
about an issue, stereotyped attitudes appear 
to break down and become individualized.* 

The urban group was significantly more in 
favor of taxing all the people to plant new 
forests (attitude object 4) on the pre-test 
and on the subsequent tests. 

Measurement of attitude toward draining 
swamps (attitude object 5) revealed no sig- 
nificant differences in average attitude be- 
tween the rural and urban groups on the pre- 
test. On the post-test, October and January 
tests there was a statistically significant dif- 
ference in average attitude with the rural 
group more disposed to drain swamps than 
the urban group, although both were sig- 
nificantly more in favor of retaining swamps 
after presentation of the stimulus material 
(Table I) than they were before. 


*Remmers, H. H. and Whisler, L. D., “The Effects of a 
Guidance Program on Vocational Attitudes,” Bulletin of 
Purdue University, Studies im Higher Education XXXIV, 
ao Studies in Attitudes, Series 111, September, 1938, pp. 
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‘ifference in average attitude of the 
nd urban groups persists through the 
nd January tests. From some cause, 
rroup becomes more variable in its 

ve. as a group, between the post-test 
» October test, for the dispersion of the 
sroup becomes statistically — signifi- 
rger than that of the rural group. 
ition is present after the January 


\. SUMMARY AND CONCLUSIONS 


I\ 

\ total of 300 rural and urban high school 

sydents were measured, with reference to 
attitudes toward five conservation 
ne day prior to and immediately fol- 


ving the administration of stimulus mate- 
| designed to change the average attitude 
‘the group in a premeditated direction. The 
gtitudes were remeasured with comparable 
forms of the same scale after one month, four 
months and eight months (rural group only). 


Comparisons were made of shifts in aver- 
age attitudes; persistence of average attiiude; 
changes in variability as affected by the 
stimulus material and differences between 
rural and urban populations in average atti- 
tude and changeability of attitude. 

1. The attitudes of high school pupils to- 
ward certain conservation issues can be sig- 
nificantly changed in a desired direction. 

2. The attitudes of high school pupils to- 
ward certain conservation issues, having been 
changed by defined stimulus material, tend 
to persist as changed after a lapse of as much 
as eight months. 

3. The attitudes of the group were gener- 
ally less homogeneous after presentation of 
the stimulus material than before. 

4. The rural group tends to be less affected 
on the average by the stimulus material than 
the urban group. 








